1875-NEW PPSS_T023_OMBClearance_1 Part B 8 18 16

JANUARY 2015

Task Order 23: Study of School Climate Transformation Grants

Draft 1: OMB Clearance Request, Part B

Prepared for

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Policy and Program Studies Service

June 21, 2016

Contents

Page

Introduction 1

Supporting Statement for Paperwork Reduction Act Submission 1

Description of Statistical Methods (Part B) 1

B1. Sampling Design 1

B2. Procedures for Data Collection 2

B3. Methods to Maximize Response Rate 3

B4. Expert Review and Piloting Procedures 4

B5. Individuals and Organizations Involved in Project 4

Supporting Statement for Paperwork Reduction Act Submission

Description of Statistical Methods (Part B)

B1. Sampling Design

This section describes the selection criteria used to determine which grantees from the three grant programs are eligible to participate in this study. Because this study focuses on the coordination required when a site has multiple grants, states, and districts that were awarded a Department of Education (Department) School Climate Transformation Grant (SCTG) and either a Department of Health and Human Services–funded Project AWARE grant or a Department of Justice (DOJ)–funded School Justice Collaboration Program grant will be asked to participate.

Exhibits B.1 and B.2 show the number of districts, the number of states, and the local family court funded for each of the three federal grant programs. The numbers that show the overlap in each of the Venn diagrams represent the grantees that will be asked to participate in this study. As shown in Exhibit B.1, 28 local education agency grantees are coordinating both an SCTG program and a Project AWARE grant. One DOJ grantee also has an SCTG program and therefore will be invited to participate in this study.

Exhibit B.1. Overlap of Funded Sites at the District Level

28

1

2*

0

*The two sites receiving Project AWARE and School Justice Collaboration Program grants that do not also have SCTG grants are excluded from this study.

The state education agency grantees overlap on two of the federal grant programs: SCTG and Project AWARE (Exhibit B.2). A total of nine grantees will be asked to participate in this study.

Exhibit B.2. Overlap for SCTG and Project AWARE Grantees at the State Level

9

All 38 sites eligible to participate in this study will be invited to participate.

For each set of grantees, the designated federal project officer will supply the contractor with the list of project directors. The contractor will contact the project directors, explain the study, and invite participation. The study team will ask each project director to identify the following:

• The staff member who works most closely with the other grant at that site (in some instances, this will be the project director)

• One to two staff members who are actively engaged in the work of the grant (e.g., a lead teacher from a high school, an elementary school counselor, a professional development specialist)

The contractor will contact all nominated staff and invite them to participate in an interview. If a nominee declines, the contractor will ask that individual to suggest an alternate respondent. At least two and no more than four respondents will be interviewed at each grantee site.

For some grantees, the SCTG personnel may overlap with the Project AWARE personnel. No one will be interviewed twice, but the study team will ensure that we obtain multiple perspectives on coordination for each grant.

B2. Procedures for Data Collection

The data collection procedures for both extant and interview data are discussed in the following sections.

Collection of Extant Data

To minimize the burden involved with interviews, the contractor will systematically review all grantee applications and GPRA data to gather pertinent information prior to the interviews. These data will be provided to the contractor by the Department, thereby minimizing effort for the grantees. Hence, the study team is not requesting Office of Management and Budget (OMB) clearance for this component of the study.

Interviews

Grantee project directors for each grant program in the study sample, as well as any grant staff they nominate for inclusion in the study, will receive an e‑mail invitation to participate in the study, followed by a phone call to confirm. This invitation will introduce the study and provide a rationale for participation (i.e., the value of the knowledge to be gained about coordination and alignment across systems and funding streams). If the potential respondent indicates interest, contractor staff will schedule an interview. The scheduling process will include sharing the study consent form and a list of interview topics in advance of the interview. In this way, respondents may prepare and collect any helpful reference documents. Interviews will take place by telephone or videoconference. Each staff member will be interviewed individually to increase the likelihood of open and honest discussion. All interviews will be recorded and transcribed.

B3. Methods to Maximize Response Rate

Recruitment materials will include the Department’s endorsement of the study. The materials will emphasize the social incentive to respondents by stressing the importance of the data collection in providing much‑needed technical assistance and practical information to districts and schools. Although grantee staff participation in reasonable evaluation requests is an expectation of accepting the grant, any given individual respondent will have the right to decline the study invitation, and that refusal will be respected. Nevertheless, when educators and other staff are approached and given an opportunity to advance knowledge related to their work, and to reflect on their own practice, in our experience many will say yes.

Study materials will emphasize (1) the relevance of this study to the mission and day-to-day execution of their work, (2) the opportunity to shape knowledge about how to coordinate effectively across disciplines, and (3) the minimal burden associated with participation. The contractors will pilot test the recruitment and data collection instruments to ensure that they are user friendly and easily understandable, which will minimize withdrawal after the interviews have begun.

Past experience in data collection efforts has demonstrated the importance and value of building a consensus of support within grantees. Investing in relationship building with the project directors initially may minimize study refusals among nominated staff, helping to ensure smooth implementation of the evaluation. Based on prior experience collecting data through telephone interviews, the project team anticipates a response rate of close to 100 percent.

B4. Expert Review and Piloting Procedures

Although this OMB submission is open for public comment, the contractor will pilot the study recruitment materials and the interview protocols with no more than nine project directors who have worked in the professional capacities addressed in this study (multi-tiered systems of support in schools and access to mental health care in schools and communities). Study staff will ask these educators and administrators to react to (1) the interest level and clarity of recruitment materials; (2) the overall organization, flow, and length of the interview; (3) the clarity of the interview wording and language; (4) specific questions that were unclear or difficult to answer; and (5) any recommended changes.

The project team will use the responses from these pilots to modify the notification materials and interview protocols to make them as suitable as possible for the actual recruitment and interview activities. For example, in prior work, study staff have learned that some questions about the interviewee’s background may yield unfocused and lengthy responses at the expense of topics more central to the study. The piloting procedures also will help ensure that the interview does not take more than 45–60 minutes to administer.

B5. Individuals and Organizations Involved in Project

AIR is the contractor for the Study of School Climate Transformation Grants. The project director is Dr. Kimberly Kendziora, who is supported by an experienced team leading the major tasks of the project (see Exhibit B.3 for a list of key staff and responsibilities). During data collection and particularly during the initial phase of analysis, the contractors will draw on the cross‑staffing of some key members of the project, including the project director, deputy project director, and team leaders. Contact information for the individuals and organizations involved in the project is presented in Exhibit B.3.

Exhibit B.3. Organizations, Individuals Involved in Project

Organization	Role	Contact Name	Email	Telephone
AIR	Project director	Dr. Kimberly Kendziora	kkendziora@air.org	202-403-5391
AIR	Deputy director	Dr. Amy Mack	amack@air.org	202-403-5863
AIR	Senior advisor	Dr. David Osher	dosher@air.org	202-403-5373
AIR	Senior advisor	Ms. Sandy Williamson	swilliamson@air.org	202-527-4224
AIR	Research assistant	Ms. SooYun Chung	schung@air.org	202-403-5739
AIR	IDIQ contract director	Dr. Kerstin LeFloch	klefloch@air.org	202-403-5349
Ad Hoc Analytics	Coordinator	Mr. William Pate	wepate@adhocanalytics.com	202-595-3103
Ad Hoc Analytics	Researcher	Dr. Stacey Merola	stacey_merola@merolaresearch.com	703-447-7883
Ad Hoc Analytics	Researcher	Ms. Jessica Hammersla	jhammersla@ adhocanalytics.com	202-595-3103

ii