Evaluation of the ESSA Title I, Part C, Migrant Education Programs (Study Instruments)

September 5, 2017

The Study of Implementation of the ESEA Title I—Part C Migrant Education Program Serving Children of Agricultural Workers and Fishers

Draft (Revised) Appendix E- Regional Subgrant Coordinator/ Administrator of the Regional Migrant Education Program Interview Protocol

REGIONAL EDUCATIONAL SERVICE AGENCY

Contract GS-10F-0554N/BPA Order ED-PEP-16-A-0005/TO01

SRI Project P24149

Submitted to:

Joanne Bogart

Policy and Program Studies Service

U.S. Department of Education

400 Maryland Avenue, SW

Washington, DC 20202

Prepared by:

Policy Studies Associates

Leslie Anderson

SRI International

Deborah Jonas

Rebecca Schmidt

Introduction to the Interview

Introduce the interviewer(s).

Explain the purpose of the study and the topics to be covered in the interview. Interviewers will read the following statement to interview respondents at the beginning of each interview:

The Study of Implementation of the Elementary and Secondary Education Act (ESEA) Title I—Part C Migrant Education Program (MEP) is being carried out under contract for the U.S. Department of Education by Policy Studies Associates (PSA), a research organization, Arroyo Research Services (ARS), a research organization, and SRI International (SRI), an independent, nonprofit research institute (collectively “the study team”). The implementation study will describe how state and local MEP-funded programs have begun to change in response to changing requirements under ESEA as reauthorized by the Every Student Succeeds Act (ESSA) and how they are serving the needs of a changing migratory student population.

Explain the provisions for protecting respondent’s privacy. Interviewers will read the following statement to interview respondents at the beginning of each interview:

As part of the implementation study, the study team will share its findings with the U.S. Department of Education. However, the results of the interviews will be summarized across all the states, districts, and programs participating in interviews. Individuals will not be identified by name in any report, and data will be reported in a manner that does not reveal your identity or the identity of your program. However, we may use anonymous quotes to help illuminate the findings. Except for that which is already public, every effort will be made to maintain the confidentiality for all information collected and we will not provide information that identifies you or your district to anyone outside the study team, except as required by law. Participation in this study is required under Section 8306(a)(4) of the Elementary and Secondary Education Act.

Advise the respondent that the interview will last approximately 45-60 minutes.

Invite questions from the respondent.

Ask permission to audio-record the interview using the following statement:

We would like to record this conversation in order to ensure that we accurately capture your comments. If you agree, we would retain the recording only until we are able to validate the transcripts, at which point the voice recording would be destroyed. The transcripts will be destroyed at the end of the study. If at any point you would like to say something off the record, I will stop the recording. Do you agree to allow us to record the interview?

Confirm respondent’s current roles and responsibilities within the agency/organization and in terms of the MEP Program. Ask if there are others we should speak with regarding MEP.

Interview Questions

NOTE: Interviewers will ask all follow-up questions as specified in the protocol. In addition, interviewers will be directed to seek clarification of any response they judge to be vague or incomplete. The primary strategy for seeking clarification will be to ask respondents to provide specific examples to illustrate their responses. A second strategy will be to ask respondents to “say more” or to “expand on” or “explain the meaning of” a particular comment.

Program Staffing and Administration

1. What is your professional background?

   1. How long have you worked for the Regional Educational Service Agency (RESA)?

   2. How long have you worked with the Migrant Education Program (MEP) within the RESA? Outside the RESA?

   3. How many hours a week do you estimate you devote to MEP-related activities?

   4. What other responsibilities, if any, do you hold within the RESA?

2. In which of the following MEP-related activities is the RESA involved?

1. Identifying and recruiting (ID&R) migrant students (e.g., setting identification and recruitment (ID&R) policies; directly managing the process of identifying and recruiting eligible migrant students, contracting with an external provider to conduct ID&R activities, etc.)

2. Identifying and/or providing program services and supports to serve the needs of eligible migrant students and out-of-school youth (OSY)

3. Collaborating with other agencies and organizations working with migrant or other at-risk populations to serve their needs

4. Evaluating/monitoring service delivery, quality, and outcomes/impact

5. Providing professional development and support to providers (e.g., classroom teachers, school counselors, before or after school program staff, etc.) working with migratory children and youth

6. Other

3. Please describe the MEP staffing.

   1. How many RESA staff [number of staff positions and FTEs] are assigned to the MEP? How many staff are full-time on MEP?

   2. What are the administrative responsibilities and time commitments of RESA staff assigned to the MEP? [Probe for: delivering program services and supports for migrant students; collaborating with other agencies and organizations; providing professional development and support to providers and parent involvement activities; identifying and recruiting eligible migrant students; evaluating/monitoring service delivery and quality; COE review, data entry, and management; responding to state requirements, etc.]

   3. If RESA MEP staff have administrative responsibilities to other programs within the SEA, what are those other responsibilities?

4. In your opinion, is the number of MEP-funded staff sufficient to administer the MEP effectively? If so, why? If not, why not?

5. Do you contract with any outside contractors or consultants for services related to the MEP? If not, why not?

   1. [If yes:] How many outside contractors/consultants does your program work with and what services do they provide?
      [Prompt: evaluation, technical assistance, re-interviewing, service delivery plan, comprehensive needs assessment, professional development, recruiting, OSY services]

      1. Are contractors providing educational services (e.g. tutors)?

      2. What percentage of your administrative set-aside under the MEP is allocated to outside contractors/consultants?

[Note to Interviewer: If regional educational service provider (RESA) that awards subgrants, confirm the distribution of subgrant awards by size (e.g., funding allocation and number of eligible migrant students served) and type (i.e., LEAs, LOAs)]. What are the criteria your RESA uses to award subgrants to LEAs and/or LOAs?

Identification and Recruitment

6. [If indicated in Question 2 that RESA is involved in ID&R] What is the size of the eligible migrant student population in the region?

7. How does your RESA identify and recruit migrant students?

   1. How many RESA staff are involved in ID&R activities?

   2. What ID&R strategies are used? [Probe: Strategies/criteria for selecting recruiters; number of recruiters; particular regions or school districts that receive special focus/are of highest priority; strategies for identifying hard-to-reach populations]

   3. How, if at all, does your RESA work with other agencies, organizations, schools, and community-based organizations to coordinate its migrant student identification and recruitment efforts?

   4. What criteria are used for identifying eligible migrant students?

   5. Does your RESA apply any additional criteria beyond the federal criteria for determining migrant student eligibility?

      1. If so what criteria and why?

   6. Are there processes in place to ensure the accuracy of eligibility determinations?

      1. If so, what are they?

8. What changes has the RESA made to its migrant student identification and recruitment policies and practices in response to the Every Student Succeeds Acts (ESSA)? (Probe for: definitions of migratory workers and fishers; migratory child; qualifying move; PFS; OSY; funding allocations]

   1. What identification and recruitment strategies have been added or changed so far in response to ESSA—or for other reasons—and why?

      1. What were the benefits and challenges, if any, of these additions or changes?

   2. What plans, if any, does your RESA have to change its ID&R practices in the future in response to ESSA—or for other reasons—and why?

   3. Does the RESA apply any additional criteria beyond the federal criteria for determining migrant student eligibility?

9. In what ways, if at all, do identification strategies differ for migratory out-of-school-youth (OSY) (i.e., those who have already dropped out of school, are working towards their GED, or are “here to work” only) compared with migratory youth who are still in school?

10. Please describe some of the successes and challenges with ID & R activities at the regional level. In your opinion, are all or the vast majority of eligible migratory children identified and recruited in your RESA? If so, how do you know? If not, why not?

Migrant Student Needs

11. What are the most significant barriers or challenges that migrant students in your region face that may disrupt or delay their educational progress? [Probe for: cultural and language barriers, social isolation, academic achievement, educational continuity, access to services, health-related problems, family support, etc.]

12. How, if at all, have changes in migrant communities in your RESA affected the current needs of migrant students? [Probe for: small/large migrant population size, concentrated or spreading population, fluctuating migrant population numbers, funding, cross-agency collaboration]

13. Does your State’s Service Delivery Plan (SDP) align with or influence local priorities?

    1. In what ways did your program provide input on your state’s SDP?

    2. In what ways does the state’s SDP change what you do in terms of the students served and the services provided?

    3. Do you believe the MPOs focus on the appropriate outcomes for the MEP? If not, why not?

       1. What migrant student needs have been identified that the state’s SDP and MPOs do not address? Why?

       2. What MPOs do you believe are particularly challenging for your program to achieve? Why?

MEP-funded Services and Activities

NOTE TO INTERVIEWER: Request a copy of their Service Delivery Plan (SDP).

14. Please describe the types of instructional services your RESA funds or directly delivers to support the needs of migrant students.
    [Probe for: instructional services including ELA, math, and other supplemental instruction; credit recovery/online courses; guidance/advocacy services; preschool services; preparation for post-secondary transition, etc.]

    1. Who delivers these instructional services to eligible migrant students?

       1. How many school staff are involved in delivering those services

       2. To what extent have they received training in working with migrant student populations?

    2. What are the challenges, if any, to delivering these academic services to eligible migrant students?
       [Probe for: small/large migrant population size, concentrated or spread population, fluctuating migrant population numbers, funding, level cross-agency collaboration]

       1. How are these challenges addressed?

    3. To your knowledge, what benefits have resulted from the instructional services delivered to eligible migrant students? How do you know?

    4. How, if at all, are these services evaluated?

15. Please describe the types of support services your RESA funds or directly delivers to support the needs of migrant students.
    [Probe for: health, dental, and eye care; school supplies; clothing; transportation; parent education and support; referrals etc.]

    1. Who delivers these support services to eligible migrant students?

       1. How many school staff are involved in delivering those services

       2. To what extent have they received training in working with migrant student populations?

    2. What are the challenges, if any, to delivering these support services to eligible migrant students?
       [Probe for: small/large migrant population size, concentrated or spread population, fluctuating migrant population numbers, funding, cross-agency collaboration]

       1. How are these challenges addressed?

    3. To your knowledge, what benefits have resulted from the non-academic support services delivered to eligible migrant students? How do you know?

    4. How, if at all, are these services evaluated?

16. What OTHER services or activities are provided to support the needs of migratory children and youth (including OSY)? [Probe specifically for professional development and parent involvement activities] or?

    1. In what ways does your RESA fund or deliver professional development (i.e., for local program staff, school practitioners on ID&R, service delivery, and/or data collection and reporting)? How are professional development needs of staff members determined? How many staff members receive professional development?

       1. Who delivers these services?

       2. What challenges, if any, are associated with delivering these professional development services?
          [Probe for: lack of interest; lack of funding, lack of quality, etc.]

       3. How, if at all, does the RESA address these challenges?

       4. To your knowledge, what benefits have results from these professional development services and activities? How do you know?

    2. In what ways does your RESA fund or deliver parent involvement and education services and activities (e.g., adult education, ESL, GED, etc., or parent training on supporting their children’s educational growth)? How many parents participate in parent involvement and education activities?

       1. Who delivers these services?

       2. What challenges are associated with delivering these services?
          [Probe for: lack of interest; lack of funding, lack of quality, etc.]

       3. How, if at all, does the RESA address these challenges?

       4. What benefits have results from these services and activities?

17. How does the RESA ensure that MEP-funded services and supports are supplementing rather than supplanting services already delivered to migratory students?

18. Looking across the services and supports provided to migrant students, which have been the most effective in meeting migrant student needs? Why? How do you know? [Probe: better outcomes, greater satisfaction or participation, easier implementation, more cost-effective, other reason]?

19. Among the portfolio of services and supports offered to migrant students in your RESA (e.g., academic, non-academic/advocacy, health, etc.),, which, if any, do think should be added, dropped, or adjusted? Why?

    1. Are there needed services and supports that your RESA lacks the capacity to provide? If so, why?

    2. To what extent do you believe that ESSA will affect the services (e.g., type, frequency, duration, distribution, etc.) provided to eligible migrant students? If ESSA has not yet had an impact on the type and delivery of services and supports for migratory children and youth, do you anticipate that changes will come? Why? How, if at all, do you expect those changes might affect your capacity to meet the needs of eligible migrant children and youth?

20. What types of outreach materials and activities, if any, does your RESA use to encourage migratory children and their families to participate in and/or receive MEP educational, health, nutrition, and social services?

    1. Does the RESA disseminate materials to migrant children and their families to encourage participation? If so, how? If not, why not?

    2. Does the RESA sponsor recruitment activities targeted at migratory children and their families?

       1. What is the purpose of these activities?

       2. How often are they held?

    3. Who at the RESA is responsible for coordinating outreach activities?

    4. How, if at all, does the RESA coordinate outreach efforts with schools/projects?

Services for Migrant Special Populations

21. Which MEP-funded services has your RESA prioritized providing for students who have moved within the year, are at-risk of failing, or dropped out of school, or otherwise have been identified as PFS?
    [Probe for: instructional services including ELA, math, and other supplemental instruction; guidance/ advocacy services; preschool services; preparation for post-secondary transition; staff professional development, health, dental, and eye care; school supplies; clothing; transportation; parent education and support; referrals; staff professional development, etc.]

    1. How did your RESA determine those priorities?

    2. What MEP-funded services are provided to students identified as PFS compared with other migrant children and youth who are eligible to receive MEP-funded services and supports but are not identified as PFS?

    3. [For serving OSY]: How and when are services delivered to eligible migrants who are OSY? [Probe for: through what structures/personnel?] What factors dictate when these services are provided to OSY?

22. What specific instructional and support services, if any, has your RESA prioritized providing for older migrant students who are working toward high school completion, post-secondary education, and the workforce?
    [Probe for: Review and revise local attendance and credit accrual policies; professional development to educators and social service providers on the special needs of migratory youth; alternative schooling options, such as individualized instruction, credit recovery, online courses, and providing access to adult education courses; after school opportunities for supplemental instruction and enrichment; health and wellness support, including medical, dental, and mental health services; life skills development, including communications, problem solving, critical thinking, and behavior management skills.]

    1. What factors did your RESA consider when selecting those priority strategies?

    2. How and when are these services to improve high school completion rates provided to eligible migrant students (i.e., year-round, summer, school-year)?

       1. What factors dictate when these services are provided?

Coordination and Collaboration

23. Does your RESA coordinate its administrative responsibilities and/or program activities with other departments, agencies, organizations, or programs in the region to address the needs of migrant students? If so, why? If not, why not?

24. [If yes:] With what departments, agencies, organizations, and programs does your RESA coordinate and collaborate?
    [Probe for: Title I, Parts A & D; Title III; Title IV, Part B; Title VI, Part B; Education for Homeless Children and Youth Program; IDEA; Department of Health; Department of Labor; Department of Agriculture; etc.]

    1. What is the purpose of these collaborations? What are the shared goals and objectives among the collaborating agencies, organizations, and programs?

    2. Who is involved in efforts to coordinate and collaborate? What is the frequency of interactions and the topics of communication?

    3. What strategies or processes have supported effective coordination and collaboration between the RESA and other agencies, organizations, and/or programs?

    4. What are the benefits of coordination and collaboration for the MEP and for migrant students? Has it increased the types, frequency, duration, volume, and/or quality of services provided to migrant students?

    5. What are the challenges or drawbacks, if any of coordinating and collaborating with other agencies, organizations, and programs?

25. Does your RESA make referrals to other service providers to directly provide services to migrant students?

    1. [If yes] Describe the portfolio of services offered through referrals.

    2. Are there areas of service (e.g., health care, counseling, etc.) for which your RESA does not currently refer migratory students and/or OSY to other service providers? If so, why?

    3. What gaps, if any, exist in the service providers to whom the RESA can refer migratory students and/or OSY?

26. How, if at all, does your RESA coordinate and collaborate with the College Assistance Migrant Program (CAMP), the High School Equivalency Program (HSEP), and other programs funded by the U.S. Office of Migrant Education (OME)?

    1. What is the purpose for these collaborations?

    2. What are the shared goals and objectives?

    3. Who is involved in efforts to coordinate and collaborate?

    4. What is the frequency of interactions and the topics of communication?

    5. What strategies or processes have supported effective coordination and collaboration between RESA and CAMP, HSEP, and/or other OME-funded programs?

    6. What are the benefits of coordination and collaboration with CAMP, HSEP, and/or other OME-funded programs for the MEP and for migrant students? Has it increased the types, frequency, duration, volume, and/or quality of services provided to migrant students?

    7. What are the challenges or drawbacks, if any, of coordinating and collaborating with other OME-funded programs?

Accountability and Data Use

Technical Assistance, Monitoring, and Accountability

27. Has the state provided you with any technical assistance regarding MEP?

    1. If so, what was the nature of the assistance and how was it delivered? Was it helpful?

    2. If not, why not?

28. What other types of supports, if any, has the state MEP provided to you?
    [Probe for: statewide conferences, resource guides, etc.]

    1. To what extent did you find this support helpful? If not, why not?

29. From what other sources, if any, have you sought or received support in the administration of the MEP?

30. Does the state monitor your MEP-funded program/activities? If not, why not?

    1. If so, what specific strategies does the state use to monitor your MEP-funded program/activities?

       1. For example, do state administrators conduct in-person monitoring visits to your program? If so, how often?

    2. How, if at all, have state monitoring activities supported your efforts to administer your MEP-funded program?

    3. What changes, for example, have you made in response to monitoring and technical assistance?

31. Has your program been evaluated by an internal or external evaluator? If so, when? [NOTE TO INTERVIEWER: Obtain a copy of evaluation report]

    1. To what extent, if at all, have evaluation findings informed your efforts to administer your MEP-funded program?

Data Collection and Use

32. How are data on migrant students tracked in your state’s data system?

    1. How, if at all, does your office work with the state accountability office regarding migrant student data and formal accountability reporting?

    2. Are migrant students reported as a sub-population in formal RESA reporting?

    3. Which state data on migrant students are the most useful, and how are they useful?

    4. What have been the challenges, if any, associated with complying with state reporting requirements?

    5. What changes, if any, would you like to see in federal and/or accountability system and state reporting requirements related to migrant student data? Why?

    6. Does your RESA collect additional data that are not required by the state?

       1. Please describe the data and why they are collected.

33. Do you use outcomes data to inform ongoing decisions about changing, continuing, or discontinuing MEP-funded services and supports for eligible migrant students and OSY?

    1. [If yes:] What specific types of outcomes data are used to inform your decision making regarding the delivery of services and supports? For example, do you look at student outcomes data—academic (academic performance, graduation, grades, course completion) and non-academic (health, job attainment, housing)—to inform your decision making?

       1. Which outcomes data are most useful for purposes of informing local goals and objectives? Why?

       2. What is the process for reviewing these outcomes data for decision-making purposes, and how frequently does that review occur?

    2. What program decisions were made based on outcomes data?
       [If no:] Are there challenges or other reasons that inhibit the use of outcomes data or other evidence for program decision-making?

    3. What service-level data do you track, if any? How are those data tracked and how are they used when reviewing program outcomes?

34. What data sources, if any, does your RESA draw on to make decisions about the effectiveness of MEP services and activities for your eligible migrant student population?
    [Probe for: LEA student performance, enrollment, graduation data; participant/parent/staff surveys; individual student plans; program data, MEP participant/end-of-project reports]