Exploratory Study on the Identification of English Learners with Disabilities
Supporting Statement for Paperwork Reduction Act Submission
PART B: Description of Statistical Methods
Contract ED-PEP-11-O-0088, Task Order 2
EDICS Tracking Number 4831
OMB Number: (1875) New-XXXX
Revised XX/XX/XXXX
September 2012
Prepared for
Policy and Program Studies Service
U.S. Department of Education
Prepared by
Westat
Contents
Page
Section B: Description of Statistical Methods 1
B.1. Case Study Sampling Criteria 1
B.2. Information Collection Procedures 3
B.3. Methods to Maximize Participation 4
B.4. Tests of Procedures or Methods to be Undertaken 4
B.5. Individuals Consulted on Study Design and Involved in Project 5
List of Appendices
Appendix A: School District Introduction Letter A-1
Appendix B-1: Case Study Interview Protocol for Administrators B-1
Appendix B-2: Case Study Interview Protocol for Service Providers B-7
Appendix C: Interview Information Sheet C-1
Appendix D: Q & A about Exploratory Study D-1
Appendix E: Informed Consent E-1
Appendix F: Public Comments and the Department Responses F-1
List of Exhibits
Exhibit B-1. Sample distribution of school districts 12
Exhibit B-2. Organizations and individuals involved in the project 15
Section B. Collection of Information Employing Statistical Methods
This study is exploratory and descriptive in nature and consists of personnel interviews in six school districts across the U.S., and three schools within each district. The study is not a program evaluation and does not purport to assess program outcomes. The sample for this study will be purposively selected. Findings from the site visits will not yield any generalizable outcome since the sample is not representative.
We will use data from the Common Core of Data (CCD) Local Education Agency (LEA) Universe Survey academic years 2008–09 through 2010–11 to establish an initial sample. First, we will trim the 2010-11 data by eliminating school districts with the following characteristics: missing or no total students, fewer than 20 English learner (EL) students, no data for EL students, a reported number of EL students that is more than 100 percent of total student population, or a reported number of students with IEPs that is more than 100 percent of total student population. In addition, we will cross-reference 2010–11 data from the LEA Survey with 2010–11 data from the Public Elementary/Secondary School Universe Survey to identify and eliminate districts with fewer than three schools. Applying the above procedures to CCD LEA data from 2007–08 through 2009–10 resulted in 4,943 districts in the initial sample pool. We can expect comparable results when 2008–09 through 2010–11 data are used.
B.1. Case Study Sampling Criteria
The initial sampling pool will be coded using criteria intended to ensure diversity among case study school districts with regard to the percentage of students classified as EL, the percentage of students with IEPs, and the rate of EL population growth in the district. Specifically, we will use the following 9 sampling criteria:
Include a diversity of districts in terms of percentage of students that are ELs, excluding districts with EL populations that are too small to provide meaningful information on policies and procedures.
Include diversity of districts in terms of percentage of students with IEPs.
Include districts that have experienced substantial growth in EL population.
Include two districts that have underrepresentation of ELs in general or within particular grades or disability categories, two districts that have overrepresentation of ELs in general or within particular grades or disability categories, and two districts that have proportional rates of identification of ELs for special education.
Include at least two relatively large districts with a long history of serving EL students, and at least three districts with a relatively small, but rapidly growing population of EL students.
Include at least two districts that use English-only language instructional models, and at least two districts that use native language instructional models.
Include at least two districts that require use of RtI in the identification of specific learning disabilities, and at least two districts that use a discrepancy model.
Include at least three districts that enroll EL students from multiple language groups.
Include two urban districts, two suburban districts, and two rural districts.
First we will divide districts into quartiles using the CCD 2010-2011 variable representing the percent of students in the district classified as English learners. We will then eliminate the lowest quartile. As an example, applying that procedure to 2009–10 CCD data (range = 0.17 – 98.61%, mean= 9.59%), results in:
EL quartile A= 0.17 – 2.02% (n=1,236)
EL quartile B= 2.03 – 4.56% (n=1,236)
EL quartile C= 4.57 – 11.64% (n=1,236)
EL quartile D= 11.65 – 99.35% (n=1,235)
We will then divide the districts in EL quartiles B, C, and D into three approximately equal groups using the CCD variable representing the percentage of all students in the district in 2010–11 that have IEPs. Applying that procedure to 2009-10 CCD data (range = 0.00 – 99.62%, mean = 13.05%), results in:
IEP group 1= 0.00 – 10.75% (n=1,237)
IEP group 2= 10.76 – 14.08% (n=1,236)
IEP group 3= 14.09 – 99.62% (n=1,234)
We will then identify districts that have experienced 20 percent or more growth in their EL populations in both 2008–09 to 2009–10 and 2009–10 to 2010–11 using CCD variables that represent the number of students in a district that are identified as English learners in 2008–09, 2009–10, and 2010–11. Applying that procedure to 2009-10 CCD data, a total of 65 districts experienced 20 percent or more growth in relevant years.
Exhibit B-1 shows a sample distribution of districts across cells for criteria 1 (EL % of population) and 2 (IEP % of population), along with the subset of districts that are flagged for criteria 3 [20% or more growth in two consecutive years] based on 2007–08 through 2009–10 data.
Exhibit B-1. Sample Distribution of school districts
|
IEP Group 1 (0.00 – 10.75%) |
IEP Group 2 (10.76 – 14.08%) |
IEP Group 3 (14.09 – 99.62%) |
EL Quartile B (2.03 – 4.56%) |
232 [8] |
383 [9] |
403 [12] |
EL Quartile C (4.57 – 11.64%) |
304 [7] |
347 [4] |
347 [12] |
EL Quartile D (11.65 – 99.35%) |
271 [7] |
186 [5] |
202 [1] |
In order to obtain an eventual sample of six districts, we will randomly select four districts from each cell from an initial sample comparable to Exhibit B-1, which will result a total of 36 districts. Further, to ensure the inclusion of districts that have experienced substantial growth in EL population, one of each four districts will be randomly selected from the subset of districts flagged for criteria 3 [i.e., districts in bracket], with nine districts in total. . For example, the cell of IEP Group 1 and EL Quartile B has a total of 232 districts (this includes both substantial and non-substantial growth of EL populations) and out of which eight districts have substantial growth in EL population [i.e., the number in bracket]. To conform four districts, three districts will be randomly selected from 232 districts in this subset and one district will be randomly selected from the eight districts in the bracket.
To the extent possible, districts will be spread across states. The initial sample of 36 districts will be examined by state. For any state that is represented by more than six of the 36 districts, we will select replacement districts from within the relevant cell(s) until the sample of 36 initial districts includes no state with more than six districts.
We will then code the initial sample according to (a) representation rates of ELs in special education, (b) history of serving EL students, (c) language instructional education model, (d) use of Response to Intervention (RtI) for special education eligibility, (e) enrollment of students from multiple language groups, and (f) urbanicity of the district. We will obtain information on districts by reviewing district web sites, through review of 2010 data from Civil Rights Data Collection (CRDC) or, when necessary, establishing initial contact with districts. If, during the gathering of information related to these criteria, it appears that the randomly selected districts will not yield the intended diversity across these criteria, we will select replacement districts for the initial sample.
To recruit districts, we will send an introductory letter on Department’s letterhead with follow-up telephone calls or e-mails. Once a district agrees to participate in the study, we will work with the district contact to identify schools for participation and verify school-level data. We will select schools that are aligned with the sampling criteria characteristics of the districts, such as the proportion of ELs and students with disabilities. Within each district, we will select three schools, attempting to include both an elementary school and a middle or high school, based on the premise that processes for identifying and serving ELs with disabilities may differ across education levels.
B.2. Information Collection Procedures
This study is intended to help the Department, other policy makers, and educators better understand (a) the processes and personnel involved with identifying ELs for special education services, (b) the challenges schools and districts face in making such identifications, and (c) the strategies that schools and districts use to overcome those challenges. The study team will review previous research on identifying ELs with special needs and conduct site visits within six case study school districts. The primary source of data collected during the site visits will be interviews with district and school personnel conducted during site visits of four to five days. In addition, extant data will be collected on case study districts, including demographic information, published reports, guides, and regulations. Sources will include national data repositories; state, district, and school web sites; and documents provided by district and school personnel.
Because this study involves collection of qualitative data from a purposive sample, a discussion of statistical methodology does not apply. The study team will do everything possible to maximize the accuracy of collected qualitative data. All site visit data collectors will attend a one-day, in-depth training that familiarizes them with the instruments and trains them on interview techniques. Site visits will be conducted with two-person teams to ensure inter-rater reliability. While on-site, interviews will be recorded with permission of interviewees, in addition to notes being taken by hand. Aspects of interview notes will be sent to respondents for validation and revised as needed.
B.3. Methods to Maximize Participation
In order to maximize participation among case study districts, the U.S. Department of Education will send an introductory letter to primary contacts identified in each of the selected districts. This letter will provide information on the scope and purpose of the study and its importance for the field, what will be addressed in the interviews, issues of confidentiality, how the data will be used, who is conducting the study, and how to learn more about the study. A draft version of the introductory letter to districts with accompanying information is provided in Appendix A. In order to obtain an eventual sample of six districts, we will select a total of 36 districts for recruitment.
To recruit districts, we will send an introductory letter on Department letterhead with follow-up by telephone or e-mail. Once a district agrees to participate in the study, we will work with district contacts to identify schools for participation, verify school-level data, and identify the specific respondents within the district and schools most appropriate for participation in interviews. We will work with this individual to make certain that our visit is as efficient and non-disruptive as possible. Prior to the visit, a reminder email and/or calls will be made to the site to confirm all arrangements.
There will be one interview per respondent with minimal follow-up. Each interview should take no more than 90 minutes. No prior preparation will be needed. Questions will focus on participants’ daily work experiences. Draft versions of the case study interview protocols are provided in Appendix B.
Prior to each interview, we will send respondents an interview information sheet that provides the scope and purpose of the study, who is conducting the study, what will be addressed in the interviews, how interview data will be used, assurances of confidentiality, and contact information of the project director for questions or more information on the study. A draft version of the interview information sheet is provided in Appendix C.
B.4. Tests of Procedures or Methods to be Undertaken
In the spring and summer of 2012, the study team conducted pilot tests of the interview protocols. Six pilot tests were conducted across the service-provider and the administrator protocols. Pilot respondents included two special education teachers, a bilingual school psychologist, a district instructional coordinator, a principal, and an assistant principal. Based on results of the pilot tests, wording adjustments were made and the instruments were finalized. Following the first case study site visit, minor additional adjustments may be made to the protocols if necessary to ensure that they facilitate the acquisition of necessary information while limiting burden on respondents.
B.5. Individuals Consulted on Study Design and Involved in Project
A Technical Working Group was convened for this study to provide input on the study design, sample selection, instrumentation, data collection, and plans for data analysis. Members of this expert group included:
Sharyn Howell, Executive Director, Division of Special Education, Los Angeles Unified School District
Angelica Infante, Executive Director, Office of English Language Learners, New York City Public Schools
Janette Klingner, Professor, School of Education, University of Colorado at Boulder
Salvador Hector Ochoa, Dean & Professor, College of Education, University of Texas—Pan American
Amanda Sullivan, Assistant Professor, College of Education & Human Development, University of Minnesota
Westat is the prime contractor for this study and will carry out the study activities in collaboration with Instructional Research Group (IRG) and Compass Evaluation and Research. Key personnel include Drs. Tamara Nimkoff (Project Director), Westat; Elaine Carlson, Westat; Russell Gersten, IRG; and Anne D’Agostino, Compass. Contact information for these individuals and organizations is presented in Exhibit B-2.
Exhibit B-2. Organizations and individuals involved in the project
Organization |
Contact Name |
Telephone Number |
Westat |
Dr. Tamara Nimkoff |
919-474-0719 |
Westat |
Dr. Elaine Carlson |
757-565-4048 |
Instructional Research Group |
Dr. Russell Gersten |
714-826-9600 |
Compass Evaluation and Research |
Dr. Anne D’Agostino |
919-544-9005 |
Appendix A. School District Introduction Letter
[Date]
[Name of District Special Education or EL Director]
[Address of District Special Education or EL Director]
[Name of District Superintendent]
Re: Exploratory Study on the Identification of English Learners with Disabilities
Dear [District Special Education or EL Director],
The Policy and Program Studies Service (PPSS) of the U.S. Department of Education (the Department) would like to request your district’s participation in an important study being conducted on the identification of English learners (ELs) with disabilities. This study is designed to provide the Department, other policymakers, and educators with important information on current processes and challenges related to the identification of ELs for special education, as well as the strategies that schools and districts are using to overcome such challenges.
As part of this study, Westat is conducting site visits in six public school districts with three schools per district across the U.S. The Department requests your participation in this study through a site visit to your district between January and February 2013. Project director Dr. Tamara Nimkoff will contact you shortly to explain the site visit in detail and discuss participation of your district. For further information about the study and its data collection procedures, please consult the attached document.
Thank you very much in advance for your cooperation. We recognize that you are busy serving the students of your community and we appreciate your willingness to provide the time and expertise needed for the success of this important study. Should you have any questions about the study, please contact the Department project officer for the study, Dr. Jean Yan, at Jean.yan@ed.gov or at (202) 205-6212 or the project director for the study, Dr. Tamara Nimkoff at Westat, at tamaranimkoff@westat.com or at (919) 474-0719.
Sincerely,
Tom Weko
Director
Policy and Program Studies Service
Enclosure: Q&A about Exploratory Study
Appendix B-1. Case Study Interview Protocol for Administrators
Notes:
Interviews are expected to take no more than 90 minutes.
District Administrator respondents may include the special education director, EL director, federal programs director, assessment director, student/instructional services director, or their designees.
School Administrator respondents may include the principal, assistant principal, or department chair.
Text in [brackets] indicates instructional notes to the interviewer. Text in italics indicates actual interview questions.
Introduction
[Thank respondent for their time. Confirm that they received and had a chance to read the Interview Information Sheet.]
First, I want to thank you for taking the time to talk with us today. My name is [name] and this is [name of the other site visitor]. Before we get started, did you have a chance to read the Interview Information Sheet that provides background on this study and what we’ll be doing in the interview?
[If they did not receive/read the Interview Information Sheet, hand them a copy to keep and briefly summarize the purpose and content of the interview, and how the results will be used. Then define what is meant by English Leaners.]
For the purpose of our conversation today, when we talk about English learners, we’re referring to those students who the district has determined to be of EL status based on results of an English language proficiency (ELP) assessment. This would include students eligible for English language instructional services, as well as those students who have been reclassified as former ELs within the prior two years. Do you have any questions about the study or the interview? [Answer any questions.]
[Provide them with the Informed Consent form for their review and signature.]
Your participation in the interview and responding to individual interview questions is voluntary. You may decide not to participate or to end your participation at any time. This form outlines some of the issues in the Interview Information Sheet with regard to potential risks, benefits, and confidentiality. Please take a minute to read it and let me know if you have any questions. I’d like to ask you to sign it before we begin.
[After obtaining their signed consent form, request permission to record.]
Before we begin, I want to check with you to see if it would be okay if we record our conversation today. We will try to take careful notes as we talk, but recording the conversation will assist us making sure our notes are accurate. We will not share the recordings and notes with anyone outside of the project. Would that be okay with you?
[Once permission to record is obtained, begin recording the interview.]
Respondent Role
[The respondent’s job title will be obtained prior to interview. The question below will be asked to initiate the interview and provide context for the subsequent interview questions.]
|
English Learner Population
[Information on the district’s [school’s] EL population will have been obtained prior to site visit via CCD, district and school web sites, or direct request to district contact. The questions below on EL population should be asked only if necessary.]
[Probe: Homogeneity/heterogeneity within the EL population] |
|
Pre-referral/Early Intervening: Procedures (RQ 1)
[Any written policies of the state and district related to pre-referral/early intervening with ELs will have been obtained prior to site visit via state/district web sites or direct request to district contact. The questions below will be asked when policies/procedures are not covered in written documents or when policies/procedures require clarification.]
[Probe: Screening vision, hearing, other health issues; Observations of classroom instruction; Consideration of EL student’s achievement in relation to true peers; Consideration of student’s opportunity to learn the necessary content and skills] |
|
[Probe: Use of specially trained personnel, e.g. bilingual educators] |
[Probe: Kinds of information provided to parents; Degree of parent engagement in decision making; Stage at which parent is brought in] |
|
[Probe: Methods and process used for pre-referral decision making; Personnel involved; Involvement of parents; Time to formal special education referral; Factors to consider for ELs] |
Assessment and Identification: Procedures (RQs 1, 2, and 3)
[Any written policies of the state and district related to assessment and identification of ELs for special education will have been obtained prior to site visit via state/ district web sites or direct request to district contact. The questions below will be asked when policies/procedures are not covered in written documents or when policies/procedures require confirmation or clarification.]
|
[Probe: Tools and instruments used in assessment and identification processes] |
[Probe: IDEA 2004 exclusionary clause] |
[Probe: Re-evaluation procedures] |
[Probe: Kinds of information provided to parents; Degree of parent engagement in decision making; Stage at which parent is brought in] |
[Probe: Methods and processes used; Tools and assessment instruments used; Accommodations; Language of assessment; Involvement of parents; Factors to consider for ELs] |
[Probe: Specially trained personnel, e.g., interpreters and bilingual assessors] |
|
Assessment and Identification: Administrator Perceptions (RQs 1, 2, and 3)
|
|
|
|
|
|
|
|
|
Patterns of Special Education Identification: Administrator Perceptions (RQ 4)
[District- and school-level data on special education identification of ELs and non-ELs will have been obtained prior to site visit via district/school web sites or direct request to district contact. These will be used during the questions below.]
|
|
|
Exit from Language Instruction Education Programs: Procedures (RQ 5)
[Any written policies of the state and district related to the exiting of EL students from LIEPs as well as information on district LIEP models will have been obtained via state/district web sites or request of district contact prior to site visit. The questions below will be asked when information is not covered in written documents or requires confirmation or clarification.]
|
|
[Probe: Accommodations or modifications used to assess English language proficiency for ELs with disabilities] |
|
|
|
|
Coordination Between EL and Special Education: Administrator Perceptions (RQ 3)
[Information on district/school organizational structure (e.g., org chart) will have been obtained via district/school web sites or request of district contact prior to site visit. It may be referenced during the questions below.]
|
|
|
Additional Thoughts and Wrap-up
|
[Thank respondent again for his/her time and input, and obtain contact information for potential follow-up.]
I want to reiterate that we very much appreciate you taking the time and sharing your knowledge and experience with us today. As is mentioned in the interview information sheet, after we return home and have had a chance to review our notes from this interview, we would like to have the option of a brief follow-up with you if there is a need to check our interpretation of information you’ve provided today. Would this be okay? [If participant agrees.]
Thank you. Do you have a preference for email or phone?
[Obtain contact information for follow-up.]
Appendix B-2. Case Study Interview Protocol for Service Providers
Notes:
Interviews are expected to take no more than 90 minutes.
District Service-Provider respondents may include a district-level psychologist, speech pathologist, or instructional specialist providing support to school-based personnel.
School Service-Provider respondents may include a special education teacher, EL teacher, counselor, resource teacher, student support team leader, or general education teacher.
Text in [brackets] indicates instructional notes to the interviewer. Text in italics indicates actual interview questions.
Introduction
[Thank respondent for their time. Confirm that they received and had a chance to read the Interview Information Sheet.]
First, I want to thank you for taking the time to talk with us today. My name is [name] and this is [name of the other site visitor]. Before we get started, did you have a chance to read the Interview Information Sheet that provides background on this study and what we’ll be doing in the interview?
[If they did not receive/read the Interview Information Sheet, hand them a copy to keep and briefly summarize the purpose and content of the interview, and how the results will be used. Then define what is meant by English Leaners.]
For the purpose of our conversation today, when we talk about English learners, we’re referring to those students who the district has determined to be of EL status based on results of an English language proficiency (ELP) assessment. This would include students eligible for English language instructional services, as well as those students who have been reclassified as former ELs within the prior two years. Do you have any questions about the study or the interview? [Answer any questions.]
[Provide them with the Informed Consent form for their review and signature.]
Your participation in the interview and responding to individual interview questions is voluntary. You may decide not to participate or to end your participation at any time. This consent form outlines some of the issues in the Interview Information Sheet with regard to potential risks, benefits, and confidentiality. Please take a minute to read it and let me know if you have any questions. I’d like to ask you to sign it before we begin.
[After obtaining their signed consent form, request permission to record.]
Before we begin, I want to check with you to see if it would be okay if we record our conversation today. We will try to take careful notes as we talk, but recording the conversation will assist us making sure our notes are accurate. We will not share the recordings and notes with anyone outside of the project. Would that be okay with you?
[Once permission to record is obtained, begin recording the interview.]
Respondent Role
[The respondent’s job title will be obtained prior to interview. The question below will be asked to initiate the interview and provide context for the subsequent interview questions.]
|
Pre-referral/Early Intervening: Service-Provider Practices (RQ 1)
[These questions are intended for district- and school-level service providers whose roles have been identified as relevant to pre-referral/early intervening procedures from policy documents or administrator interview. For those respondents whose roles would not make them actively engaged in pre-referral/early intervening (e.g., school psychologists, speech pathologists) thus they would not be asked the specific questions below, simply ask them to describe what their role is, if any, during this stage.]
[Probe: Screening vision, hearing, other health issues; Observations of classroom instruction; Consideration of EL student’s achievement in relation to true peers; Consideration of student’s opportunity to learn the necessary content and skills] |
|
[Probe for district service-provider: Variation across schools in the district] |
|
[Probe: Kinds of information provided to parents; Degree of parent engagement in decision making; Stage at which parent is brought in] |
|
[Probe: Methods and process used for pre-referral decision making; Personnel involved; Involvement of parents; Time to formal special education referral; Factors to consider for ELs] [Probe for district service-provider: Variation of practices across schools in the district] |
Assessment and Identification: Service-Provider Practices (RQs 1 and 2)
[Prior to the questions below, the respondent will have been asked to have documentation accessible during the interview related to (a) one EL student who was assessed and found eligible for special education services and (b) one EL student who was assessed for special education services but found not eligible for services. Interviewers will not directly review student files; rather respondents will have files accessible for reference as needed.]
[Probe: Why was the student referred? By whom? When? What prereferral steps had been taken prior to referral? Once the child study team was convened, was there any discussion of English language proficiency v. disability? What assessments were used to assess eligibility? What were the results of the tests? Was there any doubt about the presence of a disability? Did all members of the evaluation team agree about eligibility?] |
[Probe: Why was the student referred? By whom? When? What prereferral steps had been taken prior to referral? Once the child study team was convened, was there any discussion of English language proficiency v. disability? What assessments were used to assess eligibility? What were the results of the tests? What were the reasons for ineligibility? Did all members of the evaluation team agree about ineligibility? What services have been provided to the student since the assessment?] |
|
|
[Probe: What personnel are making decisions and what is done when specially trained personnel are not available] |
[Probe: Kinds of information provided to parents; Degree of parent engagement in decision making; Stage at which parent is brought in] |
[Probe: Methods and processes used; Tools and assessment instruments used; Accommodations; Language of assessment; Personnel involved; Involvement of parents; Factors to consider for ELs] |
Assessment and Identification: Service-Provider Perceptions (RQs 2 and 3)
[Probe: Assessment of the student in both English and their first language; Supplementing formal measures with informal, contextual assessment; Use of resources from district or school] |
|
|
|
|
|
|
|
|
|
|
|
Patterns of special Education Identification: Service-Provider Perceptions (RQ 4)
[District data on special education placements of ELs and non-ELs will have been obtained prior to site visit via district/school web sites or direct request to district contact. These will be used during the questions below.]
|
|
|
Exit from Language Instruction Education Programs: Service-Provider Practices and Perceptions (RQ 5)
[These questions are intended for district- and school-level service providers whose roles have been identified as relevant to LIEP exiting procedures from policy documents or administrator interview. For those respondents whose roles would not make them actively engaged in LIEP exiting thus they would not be asked the specific questions below, simply ask them to describe what their role is, if any, in this process.]
|
|
[Probe: Modified or alternative achievement standards] |
|
|
Coordination Between EL and Special Education: Service-Provider Perceptions (RQ 3)
|
|
|
Additional Thoughts and Wrap-up
|
[Thank respondent again for his/her time and input, and obtain contact information for potential follow-up.]
I want to reiterate that we very much appreciate you taking the time and sharing your knowledge and experience with us today. As is mentioned in the interview information sheet, after we return home and have had a chance to review our notes from this interview, we would like to have the option of a brief follow-up with you if there is a need to check our interpretation of information you’ve provided today. Would this be okay? [If participant agrees.]
Thank you. Do you have a preference for email or phone?
[Obtain contact information for follow-up.]
Appendix C. Interview Information Sheet
The purpose of this study to learn more about the processes and personnel involved with identifying English Learners for special education services, the challenges that schools and districts face in making such identifications, and the strategies that schools and districts use to overcome those challenges. Westat and its partners, Instructional Research Group and Compass Evaluation and Research, are conducting the study for the U.S. Department of Education’s Policy and Program Studies Service. The study team has reviewed recent research on identifying English Learners with special needs and collected documents that describe policies and programs relevant to EL students with disabilities, and is now conducting case study site visits in six school districts and three schools in each district across the U.S.
Our purpose in these case studies is not to evaluate or monitor the practices in a specific district or school. Rather, it is to better understand the current challenges that districts and schools face in identifying ELs with disabilities and the strategies that are being used to overcome those challenges. We hope that what we learn from this exploratory study will help us generate hypotheses regarding assessing and identifying ELs with disabilities and plan a nationally representative study on the topic.
To assist with the study, we are asking educators like you, who are knowledgeable about procedures and practices for identifying ELs with disabilities, to participate in a 90-minute interview. Depending upon your role, we may ask you questions about (1) procedures, practices, and instruments used to assess and identify ELs for special education; (2) personnel involved in identification and assessment; (3) challenges related to the assessment and identification of disabilities among EL students, and strategies that are used to overcome those challenges; (4) patterns of special education identification; and (5) procedures and practices used to exit EL students with disabilities from language instruction educational programs.
Your participation in the interview as well as responding to individual interview questions is voluntary. You may decide not to participate or to end your participation at any time.
We will only use the information you provide for the purpose of this study. We will treat all information that you supply in a confidential manner, to the extent provided by law. We will not use your name and will not attribute any quotes. We also will not share what we discuss with other people outside the study team. We will not disclose the names of the districts or schools visited.
After we return home and have had a chance to review our notes from the interview with you, we may conduct a brief follow-up with you to check our interpretation of aspects of the information you provide. We can do this over email or phone.
If you have any questions or would like more information about this study, please contact ED Project Officer, Jean Yean at (202) 205-6212 or jean.yan@ed.gov or the study Project Director, Tamara Nimkoff at Westat at (919) 474-0719 or tamaranimkoff@westat.com.
Appendix D. Q & A About Exploratory Study on the
Identification of English Learners with Disabilities
What is the Exploratory Study on the Identification of English Learners with Disabilities?
Funded through Title III of the Elementary and Secondary Education Act, the purpose of this study to learn more about current processes and challenges related to identifying ELs for special education services, and the strategies that schools and districts are using to overcome such challenges. The study team has reviewed recent research on identifying ELs with special needs and is now conducting case study site visits with a sample of public school districts across the U.S.
Why is this study important?
The goal of this study is to better understand current procedures and practices used to identify ELs with disabilities, the challenges schools and districts face in making such identifications, and the strategies that schools and districts use in overcoming those challenges. Our purpose is not to evaluate or monitor the practices in a specific district or school. We hope that what we learn from this exploratory study will help the Department generate hypotheses regarding assessing and identifying ELs with disabilities and plan a nationally representative study of the these important issues.
Is participation in the study required?
Participation is voluntary.
How were districts selected?
In order to best answer the study questions, we have purposively selected six case study districts using criteria intended to ensure diversity among districts with regard to the (a) percentage of students classified as EL, (b) percentage of students with IEPs, (c) rates of special education identification, (d) history of serving EL students, (d) EL instructional models used, (e) processes for determining special education eligibility, (f) enrollment of students from multiple language groups, and (g) district urbanicity. In addition, districts are spread across states.
What are the benefits for participating?
Your participation will ensure that policymakers, researchers, and educators have access to important information on procedures and practices used to identify ELs with disabilities, the challenges in making such identifications, and the strategies that are being used to overcome the challenges.
Findings from this exploratory study will be publicly available on the U.S. Department of Education’s web site. Participating districts and schools also will receive copies of the study report. The names of the districts or schools visited will not be disclosed in any study report or public briefing.
What will the case study involve?
Your district will be visited for four to five days by a professional, experienced pair of researchers who will conduct interviews with district and school personnel who are knowledgeable about procedures and practices for identifying ELs with disabilities. Interview respondents may include:
District administrators (e.g., special education director, EL director, federal programs director, assessment director, student/instructional services director, or their designees)
School administrators (e.g., principal, assistant principal, or department chair)
District-level non-administrative personnel (e.g., psychologist, instructional specialist, and/or speech-language pathologist)
School-level service providers (e.g., special education teacher, EL teacher, counselor, resource teacher, student support team leader, and/or general education teacher)
We may also request additional information that describes policies and programs relevant to EL students with disabilities in your district.
Our study site coordinator will work to schedule our visit at a convenient time and place for the district and schools.
What kinds of interview questions will be asked?
Depending upon the respondent’s role, we will ask questions to help us better understand:
Procedures, practices, and instruments used to assess and identify ELs for special education;
Personnel involved in identification and assessment;
Challenges in the assessment and identification of EL students and strategies that are used to overcome those challenges;
Patterns of special education identification; and
Procedures and practices used to exit EL students with disabilities from language instruction educational programs.
Each interview should take no more than 90 minutes. No prior preparation is needed. Questions will focus on participants’ daily work experiences.
After we return home and have had a chance to review our notes from the interviews, we may follow-up briefly with respondents to clarify our interpretation of aspects of the information provided. Follow-up may occur over email or phone, as respondents prefer.
Will district- and school-provided information be confidential?
Yes. We will treat the information obtained in this study in a confidential manner, to the extent provided by law. We will not identify the names of individuals, or the schools or districts in which they work, in any reports or public briefings. Interview responses will be used to summarize findings in an aggregate manner that does not associate responses to a specific site or individual.
What will be done to ensure the quality of the research?
The research team will assume responsibilities designed to protect the participants and to ensure that the data collection activities are of the highest quality and lowest burden possible. These responsibilities include:
Undergoing review by the Westat Institutional Review Board (IRB);
Obtaining Office of Management and Budget clearance for the data collection. The valid OMB control number of this information collection is XXXX-XXXX; and
Presenting the study for review by an expert technical advisory panel that includes a district EL administrator, district Special Education administrator, and several nationally-known experts in English learners with disabilities.
Who is conducting the study?
The U.S. Department of Education’s Policy and Program Studies Service is overseeing this study. The following research contractors are conducting the study:
Westat (www.westat.com) leads the study with Drs. Tamara Nimkoff (Project Director) and Elaine Carlson. Westat is an employee-owned research corporation serving agencies of the U.S. Government, as well as businesses, foundations, and state and local governments. Westat conducts research studies in education, transportation, the environment, human services, health, social services, housing, energy, military human resources, and science and technology.
Instructional Research Group (www.inresg.org) is an educational research institute specializing in large-scale program evaluation field research in the areas of reading, mathematics, education of English learners, and professional development. IRG’s current projects include randomized control trials evaluating the impact of teaching English learners and comparing different models of response to intervention strategies.
Compass Evaluation and Research (www.compasseval.com) is an independent evaluation-consulting firm dedicated to working jointly with organizations to conduct evaluations that contribute to the development of successful programs. Compass has worked in the areas of early childhood; K-12 educational programs; special education; health, community, and safety; and evaluation capacity building.
For more information, please contact:
Westat Project Director: Tamara Nimkoff, (919) 474-0719 or tamaranimkoff@westat.com
U.S. Department of Education Project Officer for the study: Jean Yan, (202) 205-6212 or Jean.Yan@ed.gov
Appendix E. Informed Consent
Purpose
The purpose of this study is to learn more about current processes and challenges related to identifying English learners for special education services, and the strategies that schools and districts are using to overcome such challenges. The findings from this study will help the Department plan a nationally representative study of these important issues. We have reviewed recent research on identifying ELs with special needs and are now conducting case study site visits with a sample of school districts across the U.S.
Risks and Discomfort
There are few anticipated or known risks in participating in this study.
Benefits
Your participation will ensure that policymakers, researchers, and educators have access to important information on procedures and practices used to identify ELs with disabilities, the challenges in making such identifications, and the strategies that are being used to overcome the challenges.
Confidentiality
We will treat the information obtained in this study in a confidential manner, to the extent provided by law. We will not identify the names of individuals, or the schools or districts in which they work, in any reports or public briefings. Your responses will be used to summarize findings in an aggregate manner that does not associate responses to a specific site or individual.
More Information
If you would like more information about this study, you may contact the Project Director, Tamara Nimkoff, at Westat at (919) 474-0719 or tamaranimkoff@westat.com, or the U.S. Department of Education Project Officer for the study, Jean Yan, at (202) 205-6212 or Jean.Yan@ed.gov. For questions regarding your rights as a subject participating in this research, please contact the Westat IRB Administrator at irb@westat.com, or (301) 610-8828.
Informed Consent:
I have read the above information. I have asked questions and received answers. I consent to participate in the study.
Signature:____________________________________Date:_______________________________
Print Name:___________________________________
Position:______________________________________
District/School Name:_____________________________________
Appendix F. Public Comments and the Department Responses
Recommendation 1: The Department should increase the number of school districts to be used as case studies to ensure that a meaningful sample of policies and student populations is studied. The Department could conduct an abridged version of its study in additional districts by sending the Administrator and Service Provider interviews by mail to adjust the costs. The Department must also take affirmative steps to ensure that it chooses a diverse group of schools.
Department Response: This current study is exploratory in nature and is not intended to support inferences about the full population. Because the sample will be purposively selected and the sample will not be representative, the site visits will not yield any generalizable outcomes. As indicated in Section A.2 of the Supporting Statement, the purpose of the study is to help the Department better understand the issues to plan for a national, in-depth study of these issues.
The original sampling plan for the study included nine school districts, but we have reduced that number to six school districts based on input from the study’s technical working group (consisting of nationally recognized experts), which noted that practice can vary significantly across schools within districts, particularly medium and large school districts. As a result, we have adjusted the sampling plan to select six districts with three schools each, instead of nine districts with two schools each, to allow more variation within a district. That said, we agree that it is important for this study to select as diverse a group of districts as is possible with this small sample. Section B.1 of the Supporting Statement describes the characteristics for which we will try to diversify the sample to the extent feasible.
Available resources for this small exploratory study do not allow the expansion of the data collection to a larger set of districts, and we believe it would be more appropriate to do that in a larger, future study that is designed to be nationally representative. In addition, the protocols to be used for the case studies are designed to be conducted through semi-structured interviews on site and not as self-report questionnaires.
Recommendation 2: The Department should be sure to choose districts whose patterns of disproportional representation of ELs with disabilities fall on both ends of the spectrum. By choosing at least one district with relatively extreme, persistent under-representation; at least one district with significant, prolonged over-representation; and at least one district with a history of delayed identification, the Department may be able to shed light on the comparative practices and circumstances that lead to the wide range of disproportionality.
Department Response: As discussed above, this exploratory study will not be able to draw causal conclusions or generalizable findings about practices and circumstances that lead to disproportionality. However, we agree that it is important for this study to include districts with both underrepresentation and overrepresentation of ELS with disabilities - see sampling criterion #4 in Section B.1 of the Supporting Statement.
Recommendation 3: Given the current racial, ethnic, and linguistic diversity reflected within the total ELL category, the Department should ensure the case study school districts and schools within that reflect a large percentage of minority and EL students. We also recommend that the Department choose a geographically diverse cross-section of school districts for the case studies. The districts chosen should not only span multiple states, but also be located in a mix of urban, suburban, and rural areas. Moreover, the school districts selected for the case study should reflect current immigration settlement patterns across the U.S.
Department Response: It is important to note that a small sample of six school districts is limited in its ability to include districts with diverse characteristics on a large set of criteria and that the sample will not be representative of districts and schools with these characteristics. Nonetheless, the sampling plan does seek to include districts that vary by EL concentration (criterion #1), recent growth in EL population (criterion #3), urbanicity (criterion #9), and state.
Recommendation 4: To study the effects of educational reform resulting from legal action under IDEA and similar disability rights laws, the Department should choose districts in which IDEA class action litigation on behalf of ELs with disabilities – either in whole or in part – has led to the implementation of remedial policies and district-wide reform. It is also recommended that the Department choose to study districts in which the Office of Civil Rights (OCR) and/or the Office of Special Education and Rehabilitative Services (OSERS) have previously investigated including those related to ELs with disabilities.
Department Response: It is beyond the scope of this study to determine the effects of legal action on the implementation of remedial policies and district-wide reform. The purpose of this study is to explore the processes and personnel involved with identifying ELs for special education services in order to plan for a national, in-depth study of these issues. Therefore, we do not consider it relevant to include districts involved in past legal actions under IDEA and similar disability rights laws as a sampling criterion.
Recommendation 5: To gauge how the development of state-led assessment consortia are affecting school practices with respect to ELs with special needs, the Department should choose to study districts that are members of the Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium, as well as a state that is a member of neither organization.
Department Response: It is beyond the scope of this study to gauge the impact of the development of state-led assessment consortia on school practices with respect to ELs with special needs, so we do not think it is relevant to add membership in these consortia as a sampling criterion.
Recommendation 6: Review recent ELL with disabilities research for and conduct the case study collection (and future information collections) at the most disaggregated level possible. Data for this project should be disaggregated by:
1. small units of geography within a local education agency (LEA), to the greatest extent possible;
2. disability category;
3. ELL “sub-subgroup” category, such as:
a. recently-arrived ELLs (ELLs who have been in the U.S. school system for less than twelve months);
b. late-arrive ELLs (ELLs who enter the U.S. school system at 9th grade or above);
c. interrupted ELLs (ELLs who have left the U.S. school system and then re-entered);
d. long-term ELLs, former ELLs (ELLs who have been identified as such for at least five years);
e. former ELLs (ELLs who have exited the category within the last three years);
4. ELL’s native language(s) and home language(s); and
5. type of program and instruction implemented for an ELL.
Department Response: An early component of this study was a review of recent research on the identification of ELs with special needs. This review focused on research published from 2001 to the present, but also included some earlier studies that were particularly relevant to the topics for this current study. This review provides a theoretical and empirical foundation for the case study component, especially for the development of case study interview protocols.
We do intend to collect and examine information from the case study sites across a number of disaggregated levels, including, but not limited to, student grade level (or grade-level groupings), disability category, level of English language proficiency, and the language instruction education program model used in the district. However, it is important to note that the study will not provide generalizable findings about these groups and categories, but rather is intended to inform the design of a future, larger-scale study. Please see Section B.1, Appendixes B-1 and B-2 of this package for details.
Recommendation 7: To enhance the utility of the information to be collected, it is recommended the proposed case studies of school districts include the following data:
Number of specialists with credentialing and/or teaching degrees in both
English-as-a-second-language (ESL) or English for speakers of other languages
(ESOL), or bilingual education and special education
Current practices used to diagnose and refer ELLs with disabilities
Current practices used to identify and refer ELLs who are gifted and talented
Professional development or resources available to educators that address language acquisition and special needs
Department Response: We will include current practices used to diagnose and refer ELs with disabilities and the professional development provided to staff involved in identifying English learners with disabilities in the collection and analysis of data. Service providers’ credentials will be part of the professional development. Questions about identification and assessment for gifted and talented programs are beyond the scope of the study. Please review Appendices B-1 and B-2 of this package for further details on the data to be collected during case study interviews.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Abt Single-Sided Body Template |
| Author | FaheyE |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |