Evaluation of the ESSA Title I, Part C, Migrant Education Programs (Study Instruments)

September 5, 2017

The Study of Implementation of the ESEA Title I—Part C Migrant Education Program Serving Children of Agricultural Workers and Fishers

Revised Draft OMB Package: Appendix D — State Director/Administrator of the Migrant Education Program Interview Protocol

Contract GS-10F-0554N/BPA Order ED-PEP-16-A-0005/TO01

SRI Project P24149

Submitted to:

Joanne Bogart

Policy and Program Studies Service

U.S. Department of Education

400 Maryland Avenue, SW

Washington, DC 20202

Prepared by:

Policy Studies Associates

Leslie Anderson

SRI International

Deborah Jonas

Rebecca Schmidt

Introduction to the Interview

Introduce the interviewer(s).

Explain the purpose of the study and topics to be covered in the interview. Interviewers will read the following statement to interview respondents at the beginning of each interview:

The Study of Implementation of the Elementary and Secondary Education Act (ESEA) Title I—Part C Migrant Education Program (MEP) is being carried out under contract for the U.S. Department of Education by Policy Studies Associates (PSA), a research organization, Arroyo Research Services (ARS), a research organization, and SRI International (SRI), an independent, nonprofit research institute (collectively “the study team”). The implementation study will describe how state and local MEP-funded programs have begun to change in response to changing requirements under ESEA as reauthorized by the Every Student Succeeds Act (ESSA) and how they are serving the needs of a changing migratory student population.

Explain the provisions for protecting respondent’s privacy. Interviewers will read the following statement to interview respondents at the beginning of each interview:

As part of the implementation study, the study team will share its findings with the U.S. Department of Education. However, the results of the interviews will be summarized across all the states, districts, and programs participating in interviews. Individuals will not be identified by name in any report, and data will be reported in a manner that does not reveal your identity or the identity of your program. However, we may use anonymous quotes to help illuminate the findings. Except for that which is already public, every effort will be made to maintain the confidentiality for all information collected and we will not provide information that identifies you or your district to anyone outside the study team, except as required by law. Participation in this study is required under Section 8306(a)(4) of the Elementary and Secondary Education Act.

Advise the respondent that the interview will last approximately 45–60 minutes.

Invite questions from the respondent.

Ask permission to audio-record the interview using the following statement:

We would like to record this conversation in order to ensure that we accurately capture your comments. If you agree, we would retain the recording only until we are able to validate the notes, at which point the voice recording would be destroyed. The transcripts will be destroyed at the end of the study. If at any point you would like to say something off the record, I will stop the recording. Do you agree to allow us to record the interview?

Confirm respondent’s current roles and responsibilities within the agency/organization and in terms of the MEP Program. Ask if there are others we should speak with regarding MEP.

Interview Questions

NOTE: Interviewers will ask all follow-up questions as specified in the protocol. In addition, interviewers will be directed to seek clarification of any response they judge to be vague or incomplete. The primary strategy for seeking clarification will be to ask respondents to provide specific examples to illustrate their responses. A second strategy will be to ask respondents to “say more” or to “expand on” or “explain the meaning of” a particular comment.

Program Staffing and Administration

1. What is your professional background?

   1. How long have you worked for the SEA?

   2. How long have you worked with the Migrant Education Program (MEP)?

   3. How many hours a week do you estimate you devote to MEP-related activities?

   4. What other responsibilities, if any, do you hold within the SEA?

2. In which of the following MEP-related activities is the SEA involved?

1. Identifying and recruiting (ID&R) migrant students (e.g., setting identification and recruitment (ID&R) policies; directly managing the process of identifying and recruiting eligible migrant students, contracting with an external provider to conduct ID&R activities, etc.)

2. Identifying and/or providing program services and supports to serve the needs of eligible migrant students and out-of-school youth (OSY)

3. Collaborating with other agencies and organizations working with migrant or other at-risk populations to serve their needs

4. Monitoring and/or evaluating local MEP subgrantees

5. Providing technical assistance and professional development to MEP subgrantees

6. Other

3. [FOR MEP DIRECTOR ONLY] Please describe the MEP staffing.

   1. How many SEA staff [number of staff positions and FTEs] are assigned to the MEP? How many staff are full-time on MEP?

   2. What are the administrative responsibilities and time commitments of SEA staff assigned to the MEP?
      [Probe for: setting identification and recruitment (ID&R) policies; developing program strategies and services for migrant students; collaborating with other agencies and organizations across the state; monitoring and evaluating MEP subgrantees; providing technical assistance and professional development to MEP subgrantees; identifying and recruiting eligible migrant students; COE review; data entry and management; responding to federal requirements; etc.]

   3. If MEP staff have administrative responsibilities to other programs within the SEA, what are those other responsibilities?

4. In your opinion, is the number of MEP-funded staff and the administrative set-aside for your program sufficient to administer the MEP effectively? If so, why? If not, why not?

5. Do you contract with any outside contractors or consultants for services related to the MEP? If not, why not?

   1. [If yes:] How many outside contractors/consultants does your program work with and what services do they provide?
      [Prompt: evaluation, technical assistance, re-interviewing, service delivery plan, comprehensive needs assessment, professional development, recruiting, OSY services]

      1. Are contractors providing educational services (e.g. tutors)?

      2. What percentage of your administrative set-aside under the MEP is allocated to outside contractors/consultants?

6. [Note to Interviewer: Confirm the distribution of subgrant awards by subgrantee type (i.e., none, regional educational service providers (RESAs, LEAs, LOAs), directly to schools (Hawaii only)]. What is the rationale for structuring the MEP program this way?

   1. [If subgrantees include regional educational service agencies] Why does your state award subgrants to regional educational service providers?

   2. [If subgrantees include ONLY LEAs, not LOAs] Why does your state award subgrants only to LEAs and not LOAs or regions?

   3. [If state does not award subgrants] Why does your state not award subgrants to regional educational service agencies, LEAs, or LOAs to administer an MEP-funded program? What challenges, if any, hinder the ability of regional or local entities to deliver direct services to eligible migrant students?

   4. [If state delivers ANY direct services to eligible migrant students or their families] Why did your state choose to provide direct services to address migrant students’ needs?

   5. [If state does not deliver direct services] Are there circumstances under which you believe the state should deliver direct services to eligible migrant students? If so, what are they? What challenges hinder the state’s ability to deliver direct services to eligible migrant students and OSY? Are there instances in which you believe the state should deliver direct services, but cannot? Why?
      [Probe for: state education priorities, state vs. local capacity, grant-making strategies, etc.]

Identification and Recruitment

7. [If indicated in Question 2 that state is involved in ID&R] How and to what extent is your office involved in the identification and recruitment (ID & R) of migrant students?

[Probe for: Setting identification and recruitment (ID&R) policies; directly managing the process of identifying and recruiting eligible migrant students, contracting with an external provider to conduct ID&R activities, etc.]

1. How many state staff are involved in ID&R activities?

2. What ID & R strategies are used at the state level?
   [Probe for: Strategies/criteria for selecting recruiters; number of recruiters; particular regions or school districts that receive special focus/are of highest priority; strategies for identifying hard-to-reach populations]

3. How, if at all, does your state work with your subgrantees to coordinate identification and recruitment efforts?

4. Are there processes in place to ensure the accuracy of eligibility determinations? If so, what are they?

1. What changes has the state made to its migrant student identification and recruitment policies and practices in response to the Every Student Succeeds Act (ESSA)? (Probe for: definitions of migratory workers and fishers; migratory child; qualifying move; PFS; OSY; funding allocations]

   1. What identification and recruitment strategies has the state added or changed so far in response to ESSA—or for other reasons—and why?

      1. What were the benefits and challenges, if any, of these additions or changes?

   2. What plans, if any, does the state have to change your ID&R practices in the future in response to ESSA—or for other reasons—and why?

   3. Does the state apply any additional criteria beyond the federal criteria for determining migrant student eligibility?

2. To what extent, if at all, do identification strategies differ for migratory out-of-school youth (OSY) (i.e., those who have dropped out of school, are working towards their GED, or are “here to work” only) compared with migratory youth who are still in school?

3. Please describe some of the successes and challenges with ID & R activities at the state level. In your opinion, are all or the vast majority of eligible migratory children identified and recruited in your state? If so, how do you know? If not, why not?

Migrant Student Needs

11. What are the most significant barriers or challenges of migrant students in your state that may disrupt or delay their educational progress?
    [Probe for: cultural and language barriers, social isolation, academic achievement, educational continuity, access to services, health-related problems, family support, etc.

12. How, if at all, have changes in migrant communities in your state affected the current needs of migrant students?
    [Probe for: small/large migrant population size, concentrated or spreading population, financial necessity to stay in one location, fluctuating migrant population numbers, funding, state-level cross-agency collaboration]

13. [Note to Interviewer: confirm the date from the last Comprehensive Needs Assessment (CNA) the study team has on file] Did you participate in the process of developing the Comprehensive Needs Assessment (CNA)? Please describe the process of developing the CNA.

    1. What stakeholder groups (other state-level agencies and organizations, MEP subgrantees, community-based organizations, parents, etc.), if any, were involved in identifying migrant student needs and/or writing the CNA? Were there any stakeholder groups or individuals who were not involved in the CNA who should have been? Why?

    2. How were migrant student needs identified and prioritized? What methods were used to gather and analyze data and make decisions? What data were most influential?

MEP-funded Services and Supports

NOTE TO INTERVIEWER: Request a copy of their Service Delivery Plan (SDP).

14. Please describe the types of instructional services your state funds or directly delivers to support the needs of migrant students.
    [Probe for: instructional services including ELA, math, and other supplemental instruction; credit recovery/online courses; guidance/advocacy services; preschool services; preparation for post-secondary transition, etc.]

    1. Who delivers these instructional services to eligible migrant students? How many MEP-funded staff are involved in delivering those services and to what extent have they received training in working with migrant student populations?

    2. What are the challenges, if any, to delivering these academic instructional services to eligible migrant students and how are they addressed?
       [Probe for: small/large migrant population size, concentrated or spread population, fluctuating migrant population numbers, funding, state-level cross-agency collaboration]

    3. To your knowledge, what benefits have resulted from the instructional services delivered to eligible migrant students? How do you know?

    4. How, if at all, are these services evaluated?

15. Please describe the types of support services your state funds or directly delivers to support the needs (of migrant students.
    [Probe for: health, dental, and eye care; school supplies; clothing; transportation; parent education and support; referrals etc.]

    1. Who delivers these support services to eligible migrant students? How many school staff are involved in delivering those services and to what extent have they each received training in working with migrant student populations?

    2. What are the challenges, if any, to delivering these support services to eligible migrant students and how are they addressed?
       [Probe for: small/large migrant population size, concentrated or spread population, fluctuating migrant population numbers, funding, state-level cross-agency collaboration]

    3. To your knowledge, what benefits have resulted from the support services delivered to eligible migrant students? How do you know?

    4. How, if at all, are these services evaluated?

16. What OTHER services or activities are provided to support the needs of migratory children and youth (including OSY)? [Probe specifically for professional development and parent involvement activities]

    1. In what ways does the state fund or deliver professional development (i.e., for state staff, local program staff, school practitioners on ID&R, service delivery, and/or data collection and reporting)? How are professional development needs of staff members determined? How many staff members receive professional development?

       1. Who delivers these services?

       2. What challenges, if any, are associated with delivering these services?
          [Probe for: lack of interest; lack of funding, lack of quality, etc.]

       3. How, if at all, does the state address these challenges?

       4. To your knowledge, what benefits have resulted from these professional development services and activities? How do you know?

    2. In what ways does the state fund or deliver parent involvement and education services and activities (e.g., adult education, ESL, GED, etc., or parent training on supporting their children’s educational growth)? How many parents participate in parent involvement and education activities? Who delivers these services?

       1. What challenges are associated with delivering these services? [Probe for: lack of interest; lack of funding, lack of quality, etc.]

       2. How, if at all, does the state address these challenges? What benefits have resulted from these services and activities?

17. How does the state ensure that MEP-funded services and supports are supplementing rather than supplanting services already delivered to migratory students?

18. Looking across the services provided to migrant students (i.e., directly or through subgrantees), which have been the most effective in meeting migrant student needs? Why? How do you know?
    [Probe: improved student outcomes; greater satisfaction or participation among migratory children and youth; easier implementation; more cost-effective; other reason]?

19. Among the portfolio of services and supports offered to migrant students in your state (e.g., including academic, non-academic/advocacy, health, etc.), which, if any, do you believe should be added, dropped, or adjusted? Why?

    1. Are there needed services and supports that the state lacks the capacity to provide? If so, why?

    2. In what way(s) do you believe that requirements under ESSA will affect the services (e.g., type, frequency, duration, distribution, intensity, etc.) provided to eligible migrant students? If ESSA has not yet had an impact on the type and delivery of services and supports for migratory children and youth, do you anticipate that changes will come? Why? How, if at all, do you expect those changes might affect your capacity to meet the needs of eligible migrant children and youth?

20. What types of outreach materials and activities does your state use to encourage migratory children and their families to participate in MEP educational, health, nutrition, and social services?

    1. Does the state disseminate materials to migrant children and their families to encourage participation? If so, how? If not, why not?

    2. Does state sponsor recruitment activities targeted at migratory children and their families?

       1. What is the purpose of these activities?

       2. How often are they held?

    3. Who at the state is responsible for coordinating outreach activities?

    4. How, if at all, does your state coordinate outreach efforts with its subgrantees?

Services for Migrant Special Populations

21. Which MEP-funded services, if any, has your state prioritized providing for students who are identified as PFS?

[Probe for: instructional services including ELA, math, and other supplemental instruction; guidance/advocacy services; preschool services; preparation for post-secondary transition; staff professional development, health, dental, and eye care; school supplies; clothing; transportation; parent education and support; referrals; staff professional development, etc.]

1. How did your state determine those priorities?

2. What MEP-funded services are provided to students identified as PFS compared with other migrant children and youth who are eligible to receive MEP-funded services and supports but are not identified as PFS?

3. [For serving OSY]: How and when are services delivered to eligible migrants who are OSY? [Probe for: through what structures/personnel?] What factors dictate when these services are provided to OSY?

1. For what specific instructional and support services for older migrant students (i.e., who are working toward high school completion, post-secondary education, and the workforce) has the state funded, directly provided, or set policy?
   [Probe for: Review and revise state and local attendance and credit accrual policies; professional development to educators and social service providers on the special needs of migratory youth; alternative schooling options, such as individualized instruction, credit recovery, online courses, and providing access to adult education courses; after school opportunities for supplemental instruction and enrichment; health and wellness support, including medical, dental, and mental health services; life skills development, including communications, problem solving, critical thinking, and behavior management skills.]

   1. What factors did your state consider when selecting those priority strategies?

   2. How and when are these services to improve high school completion rates provided to eligible migrant students (i.e., year-round, summer, school-year)? What factors dictate when these services are provided?

Coordination and Collaboration

23. Does the state MEP coordinate its administrative responsibilities and/or program activities with other departments, agencies, organizations, or programs in the state to address the needs of migrant students? If so, why? If not, why not?

24. [If yes:] With what departments, agencies, organizations, and programs does the state MEP coordinate and collaborate?
    [Probe for: Title I, Parts A & D; Title III; Title IV, Part B; Title VI, Part B; Education for Homeless Children and Youth Program; IDEA; Department of Health; Department of Labor; Department of Agriculture; etc.]

    1. What is the purpose of these collaborations? What are the shared goals and objectives among the collaborating agencies, organizations, and programs?

    2. Who is involved in efforts to coordinate and collaborate; what is the frequency of interactions and the topics of communication?

    3. What strategies or processes have supported effective coordination and collaboration between the MEP and other agencies, organizations, and programs?

    4. What are the benefits of coordination and collaboration for MEP and for migrant students? Has it increased the types, frequency, duration, volume, and/or quality of services provided to migrant students?

    5. What are the challenges or drawbacks, if any, of coordinating and collaborating with other agencies, organizations, and programs?

25. How, if at all, does your state coordinate and collaborate with the College Assistance Migrant Program (CAMP), the High School Equivalency Program (HSEP), and other programs funded by the U.S. Office of Migrant Education (OME)?

    1. What is the purpose of these collaborations? What are the shared goals and objectives?

    2. Who is involved in efforts to coordinate and collaborate; what is the frequency of interactions and the topics of communication?

    3. What formalized agreements, such as Memoranda of Understandings or cost-sharing agreements, exist between the MEP and CAMP, HSEP, MEESP, and/or other OME-funded programs?

    4. What strategies or processes have supported effective coordination and collaboration between the MEP and CAMP, HSEP, and/or other OME-funded programs?

    5. What are the benefits of coordination and collaboration with other OME-funded programs for the MEP and for migrant students? Has it increased the types, frequency, duration, volume, and/or quality of services provided to migrant students?

    6. What are the challenges or drawbacks, if any, of coordinating and collaborating with other OME-funded programs?

26. In what ways, if at all, does the state MEP facilitate coordination and collaboration between subgrantees and other local programs and agencies working with migrant or other at-risk student populations? For what purposes does the state facilitate program coordination and collaboration at the local level?

1. To your knowledge, what are the challenges to subgrantees coordinating and collaborating with other organizations and agencies to serve the needs of migrant students?

27. In what ways, if at all, does the state MEP coordinate and collaborate with other states that are working with migrant or other at-risk student populations? For what purposes does the state coordinate and collaborate with other states?

Accountability and Data Use

Technical Assistance, Monitoring, and Accountability

28. Are migrant students included as a subgroup in your state accountability framework under ESSA or in your NCLB waiver (if your state has one)? If not, why not? If so, what implications has that had for the MEP?

29. In what ways do the data requirements of the federal Comprehensive State Performance Report (CSPR) adequately capture useful data regarding migrant student needs and outcomes?

    1. Which CSPR data on migrant students are the most useful, and how are they useful?

    2. Does your state collect additional data that is not required by the CSPR? Please describe the data and why they are collected.

    3. What have been the challenges, if any, associated with completing the MEP portion of the CSPR?

    4. What changes, if any, would you like to see in federal CSPR reporting requirements related to migrant student data? Why?

30. [If state has MEP subgrantees] Does the state provide targeted technical assistance regarding MEP to it subgrantees? If so, how does the state identify which subgrantees need assistance and how the state can best help?

31. What other types of supports does the state MEP provide to its subgrantees?
    [Probe for: statewide conferences, resource guides, etc.]

32. To your knowledge, has state support affected subgrantees’ efforts to deliver services and supports to eligible migrant students? If so, what? Please describe the successes and challenges resulting from state support.

    1. What are the most significant challenges associated with providing technical assistance and support to MEP subgrantees?

33. Does the state monitor its MEP-funded subgrantees? If not, why not?

    1. [If yes:] What specific strategies does the state use to monitor MEP-funded subgrantees and why? For example, do state administrators conduct in-person monitoring visits to subgrantee sites? If so, how often? Are monitoring visits conducted annually to all subgrantee sites? If not, why not?

    2. What subgrantee monitoring strategies have been most and least effective and why?

    3. Is MEP subgrantee monitoring coordinated or integrated with other ESSA program monitoring?

    4. How often are subgrantees monitored?

34. Has the state evaluated (directly or through a grant or contract with an external evaluator) the effectiveness of its MEP program? If so, how many evaluations have been conducted since 2010?

    1. To what extent, if at all, have evaluation findings informed state efforts to guide and assist subgrantees?

    2. What have been the challenges, if any, associated with conducting an MEP evaluation?

Data Collection and Use

35. What data, if any, are collected and reported on migrant students in your state’s student data system?

    1. Are migrant students reported as a sub-population in formal reporting, such as in a state report card? If not, why not?

    2. Which state data on migrant students are the most useful, and how are they useful?

    3. To what extent, if at all, does your office/MEP-funded state administrators work with the state accountability office regarding migrant student data and formal accountability reporting? What are the challenges?

    4. What changes, if any, would you like to see in the state accountability system and reporting requirements related to migrant student data? Why?

36. Does the state MEP use outcomes data to inform ongoing decisions about SDPs, such as changing, continuing, or discontinuing services and supports for eligible migrant students and OSY?

    1. [If yes:] What specific types of outcomes data are used to inform SDPs? For example, does the state look at student outcomes data—academic (academic performance, graduation, grades, course completion) and non-academic (health, job attainment, housing)—to inform SDPs? [If yes:] Does the state link individual MEP participation data to student outcomes data?

       1. Which outcomes data are most useful for purposes of informing state SDPs? Why?

       2. What is the process for reviewing these outcomes data for decision-making purposes, and how frequently does that review occur?

       3. What program decisions were made based on outcomes data?

    2. [If no:] Are there challenges or other reasons that inhibit the use of student outcomes data or other outcomes evidence for decisions about SDPs?

37. What data sources, if any, does the state draw on to make decisions about the effectiveness of MEP services and activities at the state and local level?
    [Probe for: ongoing implementation decisions, not just while crafting the SDP; SEA/LEA student assessment data, enrollment, graduation data; participant/parent/staff surveys; individual student plans; program data, MEP participant/end-of-project reports]

    1. How does the state use evaluations of program implementation and quality, either formally or informally, to make decisions about MEP implementation? To what extent, if at all, have evaluation findings informed state efforts to guide and assist local programs?

    2. Which other data sources have been most influential on decision making? Do you have a specific example of a time when those data have informed decision making or supported an activity? Who was involved and what implementation decisions were made?

38. In what ways, if at all, do federal reporting requirements through the Migrant Student Information Exchange (MSIX) facilitate migrant student records transfer, enrollment, placement, and credit accrual? What are the benefits of the MSIX? What are the challenges?