Magnet Schools Assistance Program

FY 2018 and FY 2017

MSAP External Evaluator Survey

(Web-based)

Public Burden Statement:

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is 1880-0542. Public reporting burden for this collection of information is estimated to average 15 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this individual collection, or if you have comments or concerns regarding the status of your individual form, application or survey, please contact Vicki Robinson, program manager of the Magnet Schools Assistance Program via the program inbox at Msap.Team@ed.gov directly.

Thank you for participating in this survey. It will take 15 minutes. We are assessing the FY 2017 and FY 2018 Magnet Schools Assistance Program (MSAP) grantees’ technical assistance needs, and this survey is part of our data collection effort. There are no right or wrong answers. We are interested in gathering information on grantees’ project implementation and management needs, and content interests for the 2019 Project Directors Meeting, to provide them with the most relevant and focused technical assistance.

Name of Evaluator’s Grantee:

1. Based on your observations of and interactions with your MSAP grantee, in what stage of magnet implementation are most of the MSAP schools?

• Planning

• Early implementation

• Full implementation

2. How would you characterize your MSAP grantee’s knowledge of the following MSAP activities, and how helpful would it be for the grantee to receive technical assistance in these areas?

Knowledge

1= Limited

2= Some gaps

3= Strong

Technical Assistance

1= Not helpful

2= Somewhat helpful

3= Very helpful

MSAP activities	Knowledge			Technical assistance
	1	2	3	1	2	3
Hiring key staff for the MSAP project
Retaining key staff for the MSAP project
Providing professional learning support for the magnet school staff
Providing meaningful feedback to magnet staff
Developing indicators for fidelity of implementation
Monitoring fidelity of implementation using indicators
Assessing the integration of the magnet theme throughout the program
Assessing the magnet teachers’ instructional delivery
Understanding MSAP reporting requirements
Collecting appropriate data
Using evaluation data to improve program implementation
Establishing community partnerships
Engaging families in students’ education
Rebranding the public image of the MSAP school(s)

3. Based on your observations and interactions with your MSAP grantee, what are the grantee’s top three challenges to implementing the MSAP project?

4. How would you characterize your knowledge and experience in the following MSAP activities, and how helpful would it be for you to receive technical assistance in these areas?

Knowledge

1= Limited

2= Some gaps

3= Strong

Technical Assistance

1= Not helpful

2= Somewhat helpful

3= Very helpful

MSAP activities	Knowledge			Technical assistance
	1	2	3	1	2	3
Understanding MSAP annual performance reporting requirements
Understanding the information you will collect each year to evaluate the grantee’s implementation progress and outcomes
Developing appropriate data collection plans
Establishing appropriate data collection processes
Providing the grantee with recommendations, insights, and strategies for improvement
Developing indicators for fidelity of implementation
Assessing the MSAP project’s fidelity of implementation
Planning the grantee’s impact study
Conducting the grantee’s impact study
Reporting on the grantee’s impact study

5. What are the top three challenges to evaluating the MSAP project?

2019 Project Directors Meeting Interests

8. What session topics do you think would be helpful to your grantee at the 2019 MSAP Project Directors Meeting? (Please select your top six options.)

• Implementing a strategic enrollment management approach

• Implementing student selection strategies

• Retaining students to graduation

• Engaging families in their students’ education

• Establishing long-term community partnerships

• Developing diverse and equitable instructional practices

• Integrating the magnet theme across the school

• Developing theme-based curriculum

• Implementing inquiry-based instructional methods

• Sustaining the magnet program(s)

• Improving student achievement

• Meeting minority group isolation targets

• Grant management best practices

• Project management

• Effective technology use

• Building diverse staff

• Conducting an impact study

• Annual performance reporting

• Performance management

• Other (A textbox will display as a comment field with “Please specify.”)

8. Who would you prefer be the keynote speaker at the 2019 MSAP Project Directors Meeting?

• District superintendent

• Magnet expert

• Education expert

• U.S. Department of Education representative

• Other (A textbox will display as a comment field with “Please specify.”)

8. Would a session on conducting impact studies be helpful?

• No

• Yes

11. Please indicate the top three content areas you want the impact study session(s) to cover.

12. Please indicate your preferred type of conference session for an evaluation session by ranking the items below from 1 to 3, with 1 representing least preferred session type and 3 representing most preferred session type.

• Breakout content-based presentation presented by an evaluation expert

• Breakout question and answer session with an evaluation expert

• Individual consultations with an evaluation expert

Thank you for your participation.

Glossary of Terms

Planning—Planning is a disciplined effort that produces fundamental decisions and actions that shape and guide what the magnet school will be, who it will serve, what it will do, and why it will do it. Effective planning articulates what a magnet school will achieve, the actions and resources needed to make progress, and how the school will know if it is successful.

Early implementation—Early implementation is the start-up stage of magnet school development. It is the process by which the school changes its organizational strategies, operational methods, technologies, and culture to prepare for magnet program delivery and to achieve the desired change. Early implementation involves installing systems, procedures, and new practices that build and improve the school’s capacity. In addition, some program activities will need to occur as prerequisites to implementing the complete program, and some incremental activities will be initiated.

Full Implementation—Full implementation is the operational stage of magnet school development. This is when the infrastructure, personnel, and school culture function well as a whole to support magnet program delivery. During the full implementation stage, the new theme-based magnet strategies, activities, and practices become completely integrated into the school’s organizational structures and services.

MSAP External Evaluator Survey 7