Analysis of Bullying Laws and Policies
Supporting Statement for Paperwork
Reduction Act
Part B: Statistical Methods
Prepared for:
U.S. Department of Education
Office of Planning, Evaluation, and Policy Development
Policy and Program Studies Service
Prepared by:
Evaluation, Management and Training Associates (EMT), Inc.
771 Oak Avenue Parkway, Suite 2
Folsom, CA 95630
Respondent Universe and Sampling Methods 2
Procedures for the Collection of Information 3
Methods to Maximize Response Rates and Deal with Non-response 4
Tests of Procedures or Methods to Be Undertaken 5
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 5
Federal Register 60-Day Notice (Placeholder)
Federal Register Comments (Placeholder)
IRB Approval Letter (Placeholder)
Chief State School Officer, District Superintendent, and Principal Invitation Letters
Summary of School Arrangements Form
Requested Materials Checklist
School and District Personnel Consent Form
Site Visit Interview Protocol: State Education Agency (SEA) Representative
Site Visit Interview Protocol: District Representative
Site Visit Interview Protocol: School Administrator– Student Conduct
Site Visit Interview Protocol: School Administrator– Policy Development and Programming
Site Visit Interview Protocol: Teacher
Site Visit Interview Protocol: Special Education Teacher
Site Visit Interview Protocol: Physical Education Teacher
Site Visit Interview Protocol: Transportation Personnel
Site Visit Interview Protocol: School Psychologist/Counselor
Site Visit Interview Protocol: School Resource Officer (SRO)
Site Visit Interview Protocol: Yard Supervisor
District and School Personnel Survey Forms
The sampling approach for the bullying implementation study is designed to support the key study objectives: document and describe the relation of state bullying legislation and policy to district bullying policies and the implementation of bullying policies and practices in specific schools; to determine the factors that facilitate or impede implementation (e.g., legislation, state and local policies, and school contextual factors); and to identify lessons from the field that can inform the development of school-based policy and practices to promote positive climate and reduce bullying behavior. We have chosen a sampling process that identifies 24 school sites in 12 school districts that are diverse with regard to geographic region and district characteristics. Districts and school sites will be chosen in four U.S. states selected based on the scope and detail of their state legislation.
Sampling Procedures
The sampling procedure will include three steps: (1) identifying states represented in the study, (2) selecting participating school districts and middle school sites, and (3) selecting interview respondents from among school site personnel.
State selection
The sample of states will represent the regional diversity of the U.S. (i.e., West, South, Midwest, and East) and will vary with respect to their legislative and policy environments addressing bullying. Findings from the forthcoming Analysis of State Bullying Laws and Policies report will be used to identify potential states for inclusion based on a ranking of state laws according to their overall expansiveness. The primary state selection strategy will be to identify two states whose bullying statutes rated highly in the scope and detail of coverage of key components of bullying policy, one state with a mid-range rating, and one state with less expansive legislation to provide insight into bullying policy implementation in varied statutory environments.
The final selection of states will ensure that the sample produces reasonable diversity with regard to the content of legislation and the orientation of state laws and model policies based on the following specific criteria:
Balance of state and local control: States will be selected to represent a balance of state and local discretion over school district policy formulation.
Cyberbullying: State laws will be reviewed to ensure that legislation addresses cyberbullying behavior in addition to more traditional forms of bullying.
Other school climate and violence prevention resources: The selection of states will also ensure a mix of Safe and Supportive Schools (SSS) grantees and non-grantees.
Off-campus conduct and enumeration of groups: State laws will be reviewed to ensure that the selected sample represents a mix of legislation related to handling of off-campus conduct and enumeration of certain characteristics that place students at risk of being targeted by bullying.
School district selection
Within each of the four identified states, three school districts randomly will be selected from a sampling frame constructed using data from the NES Common Core of Data 2009-2010. Two school sites will be selected at random from within each district. The objective of district and school site sampling procedure is to represent diversity in district urban and rural settings and to allow for nested analyses of school sites within districts to assess school-level variation in district policy implementation. The selection of districts and school sites will involve the following steps:
Districts in each state will be sorted into three categories or sampling strata to represent diversity in urban-rural locale. These categories include city, suburban or town, and rural. One LEA will be randomly selected from within each category.
School case study sites in school districts where there are more than two 6-8 or 7-8 middle school sites will be randomly selected. School sites in districts with only two 6-8 or 7-8 middle schools will automatically be recruited into the study. Districts that do not include at least two middle school sites covering a 6-8 or 7-8 grade span will be excluded from the sample to allow for analyses of school variation within districts.
If a selected district chooses not to participate during the study recruitment period, a replacement district will be randomly selected from among the remaining districts in its urban-rural category. If a school chooses not to participate, a replacement school will be randomly selected from the remaining school sites. If one or both school sites within a district decline to participate and there are no available replacement sites, a replacement district will be selected.
Interviewee selection
Interview respondents for any position where there are more school personnel than the minimum number required to complete the full set of field interviews (e.g., teachers, yard supervisors, bus drivers) will be randomly selected from a pool of potential interviewees within each position category. At each school site, a school secretary or other administrative support person will be asked to provide lists of potential interviewees within each job category and to assist with interview scheduling for selected respondents. If there is only one individual listed within a specific position (e.g., Principal, vice-Principal, SRO), the individual will be automatically be recruited into the study.
The sampling objective for the bullying study is purposive, to select sites that show substantial variation with regard to state legislative environment and school characteristics (e.g., urban-rural locale). The resulting sample will facilitate understanding of how bullying policy is implemented under varying statutory, institutional, and community circumstances. It does not support a probability sample that would provide accurate estimates for the entire nation, states, or districts from which the sample is drawn. As the study is not designed to support statistical estimation of population parameters, power estimates are not relevant.
The procedures for the information collection include (1) procedures for obtaining cooperation from school districts and approval from schools to participate, (2) procedures for training study personnel, (3) scheduling and logistical arrangements for data collection, and (4) on-site and off-site data collection procedures for conducting interviews with state and district representatives and school site personnel.
2.1. Obtaining Cooperation from SEAs, LEAs and Schools
The process of obtaining cooperation from school districts and school sites to participate in the study will be led by the EMT Field Coordinator. The first step in the process of obtaining district and school approval is to contact a representative from the state educational agency within each selected state to encourage their commitment to the study and to request a letter of support. The second step will involve sending an invitation letter that fully explains study procedure, purpose, and benefits. The letter will be personally addressed to the school Superintendent within each district (Appendix D). If possible, we will obtain and include letters of support from relevant national and state organizations. These letters provide a strong statement about the importance of study participation. The EMT Field Coordinator will follow up with each invitee within one week in order to answer questions and to secure district approval for study participation. At this time, the Field Coordinator will identify one or two middle school sites to participate in the two-day field visits. The initial contact in a school district is addressed to the superintendent to ensure that the district puts its complete support behind the project. In turn, the superintendent can be expected to delegate responsibility for working with selected schools to coordinate study participation.
Following receipt of district approval, each potential school flagged for participation will be contacted and consent will be formally obtained, even if the district has already consulted with the school. The EMT Field Coordinator will use a streamlined approach to site recruitment by including the recruitment packets for the schools in the original recruitment package sent to the school district. This provides the district a convenient way to inform schools about the study and encourage their participation. The school site recruitment package will include a letter of invitation (Appendix D), a school site materials checklist (Appendix E), and copies of consent forms for school personnel (Appendix G). The Superintendent will be asked to write words of encouragement to the school. The Field Coordinator will maintain regular contact with the district to determine when packets have been forwarded to the schools, and to follow up directly with sites in order to obtain formal approval and to identity an appropriate school site contact. EMT’s Field Coordinator will follow-up with each contact in order to identify the list of key informants who will participate in the study, to coordinate interview scheduling, and to collect preliminary materials listed on the School Site Materials Checklist (Appendix F).
2.2. Training for Study Personnel |
No more than four months prior to initiation of the data collection, EMT will train field visitors to conduct on-site interviews. Training of field visitors will be conducted by the Principal Investigator and will include a thorough orientation to the field interview protocol, and the administration and data coding and quality assurance procedures. By the end of training, field visitors will have demonstrated that they have the professional skill set to represent EMT in the field; are knowledgeable about the study; and are committed to the project and the field team.
2.3. Scheduling and Logistical Arrangements for Data Collection
Scheduling and logistical arrangements for field visits will be handled primarily by the EMT Field Coordinator. Working collaboratively with each school’s principal or designated contact, the Field Coordinator will arrange dates for two-day site visits by two member interview teams for each site. Field visit dates are scheduled at the convenience of schools and in a way to allow for the control of travel costs, and to complete the data collection in a timely manner. Field visit scheduling will be arranged to accommodate school commitments (e.g., standardized testing dates and school events) and will be sensitive to concerns about interruptions and loss of instructional time. Field visitors will receive their assignments two to three weeks in advance with details of the data collection arrangements at each assigned school provided on the Summary of School Arrangements Form (Appendix E). Site visit and interview schedules will allow for 24 site visits to be completed within an approximate six-week period.
The Field Team Lead for each two-person team will contact the school prior to the site visit date in order to determine if there are any last-minute challenges that need to be addressed or logistical considerations that may have arisen. The lead is also responsible for preparing materials for the field visit, and facilitating data collection on the date of administration.
2.4. On- and Off-Site Data Collection
Interviews with a representative from each state education agency will be conducted by phone one to two weeks prior to scheduled field visits within that state. Each telephone interview is anticipated to be 45 minutes in length. Interviews with school district representatives will also be conducted by telephone.
Data collection for key school district and school site personnel will involve the administration of a brief 16- to 20-item, closed-ended survey questionnaire followed by a semi-structured interview process. For school district representatives, the brief survey will be administered electronically using a .pdf form that is e-mailed to the interview respondent at the time the telephone interview is scheduled. The district representative will be asked to complete the form electronically and return it to the interviewer prior to the scheduled interview time and date. For school site personnel who will be interviewed by the field team at the school site, the paper-and-pencil survey form will be administered at the beginning of the scheduled interview session and will be collected by the interviewer prior to initiating the interview process.
Field interviews with school site personnel will be conducted on-site by two trained field visitors. The team will arrive at each school on the first day scheduled and will follow any specific instructions for checking in on campus. The EMT Field Coordinator will work collaboratively with each school contact to coordinate the completion of interviews over the course of each two-day site visit.
Study participants will be distributed across two school sites within each of three school districts across the four targeted states. This sampling framework is designed to minimize the amount of burden on selected districts while allowing for a substantial breadth and diversity of school settings represented in the study. The study is not designed to support statistical estimation of population parameters, thus non-response is not a specific concern. However, procedures will be put in place to maximize response rates. Specifically, if a district declines to participate in the study, it will be replaced with another randomly selected district from within the same state and urban-rural category. Site visit schedules will be determined through a coordinated effort between the Recruitment Specialist and district and school administrators to accommodate school scheduling preferences. School personnel will be randomly selected to participate in site interviews when there are multiple personnel available to serve as respondents within a given position category. Personnel who decline to participate will be replaced with other randomly selected staff members. Once districts and school sites have formally committed to participate in the study, full participation of interview respondents is anticipated.
Interview protocols were pilot tested in July 2011 with school personnel from the Los Angeles Unified School District (LAUSD). Pilot interviews were conducted by telephone with two district representatives, one middle school Principal, two high school teachers, and one bus driver. The piloting process resulted in minor revisions to how bullying is defined within the interview context and how costs of implementation are assessed. Minor edits were made throughout the interview protocols; no major revisions were required. Information from the pilot study will be used to inform training for site visits in an effort to minimize respondent burden and maximize utility of data.
EMT Associates, Inc.
Fred Springer
771 Oak Avenue Parkway, Suite 2
Folsom, CA 95630
Phone: (916) 983-6680
Fax: (916) 983-6693
Email: fred@emt.org
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Robert Blagg |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |