1875-NEW TSLP OMB Package_Supporting Statement Part B

Contract Number: ED-PEP-11-O-0088/TO28

Task Order 28

Implementation Study of the
Turnaround School Leaders Program

OMB Package

Supporting Statement Part B

October 5, 2016

Prepared for:

U.S. Department of Education

Policy and Program Studies Service

Prepared by:

Westat

1600 Research Boulevard

Rockville, MD 20850-3129

(301) 251-1500

and

Policy Studies Associates

1718 Connecticut Avenue, NW, Suite 400

Washington, DC 20009

(202) 939-9780

U.S. Department of Education

Implementation Study of the Turnaround

School Leaders Program

SUPPORTING STATEMENT FOR

PAPERWORKREDUCTIONACT SUBMISSION

Part B: Supporting Statement For

COLLECTION OF INFORMATION EMPLOYING
STATISTICAL METHODS

CONTENTS

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 1

B.1. Respondent Universe and Sampling Procedures 1

B.2. Statistical Methods for Sample Selection 1

B.3. Methods to Maximize Response Rates 1

B.4. Pilot Testing 2

B.5. Individuals and Organizations Involved in This Project 2

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Procedures

The Implementation Study of the Turnaround School Leaders Program includes the universe of 12 Cohort 1 grantees. The study team will administer an online survey to all Cohort 1 grantees and conduct seven case study sites within the Cohort. The study team selected the case study sites using a purposive sampling approach that included reviewing grantee applications and consulting with the Department to gain a better understanding how grantees were implementing their programs. The grantees selected are both diverse across a number of factors and provide potentially fruitful comparisons with each other. Four factors were given priority during the selection process: implementation progress, grantee type, urbanicity, and project design. Respondents for the interviews and focus groups include project coordinators, key project and partner staff, and current and aspiring school leaders.

B.2. Statistical Methods for Sample Selection

No statistical methods were used to select sites for this study. Surveys will include the universe of cohort 1 grantees. The seven case study sites were selected using a purposive sampling approach that resulted in grantees that are diverse across key factors and will provide useful comparisons with each other. Following a review of grantee applications and consultation with the Department, we prioritized five factors:

• Implementation progress. The degree to which projects have encountered implementation difficulties or delays.

• Grantee type. Type of lead organization (local education agencies, institutions of higher education, state education agencies, and non-profit organizations.

• Partner type. Types of partner organizations with which the grantee is working.

• Urbanicity.

• Project design. Types of leader selection, training, and ongoing support.

B.3. Methods to Maximize Response Rates

The study team will maximize response rates by developing survey instruments that are straightforward for respondents to complete and by following up with nonresponders via email and telephone. This study is striving for a response rate of 85 percent for each group of respondents. Given the small number of sites participating in this study, our team will follow up regularly with the project directors to encourage them to complete the survey and participate in the case studies. We will stress the importance of their participation in furthering the knowledge base of effective leadership in turnaround schools and creating a pipeline for developing those leaders.

B.4. Pilot Testing

Instruments were pilot tested in September 2016. All of the instruments for this study build on prior work in the field, instilling confidence that they should glean both reliable and valid data, be straightforward to administer in the field, and present as little burden as possible on respondents. Pilot testing provides feedback to the study team on the clarity of the questions and on burden. Because of the small number of sites in the study, we piloted the instruments with two Cohort 2 grantees to determine if the questions were clear and reflected the intended meaning. Westat's Instrument Design, Evaluation and Analysis (IDEA) Services also reviewed the instruments. IDEA is a team of survey methodologists and analysts with expertise in questionnaire design, questionnaire evaluation methods, and qualitative data analysis. The instruments were revised based on the pilot and IDEA review.

B.5. Individuals and Organizations Involved in This Project

Table B-1 shows the individuals who will be responsible for data collection and analysis.

Table B-1. Individuals involved in this project

Name	Affiliation	Role
Allison Henderson	Westat	Project Manager
Dan Aladjem	Policy Study Associates	Project Lead
Anthony Milanowski	Westat	Senior Researcher
Brenda J. Turnbull	Policy Studies Associates	Senior Advisor
Ivy Morgan	ED/PPSS	COR