PEP OMB clearance request

American Institutes for Research®

Evaluation of the Carol M. White Physical Education Program -
OMB Clearance Request

May 3, 2011

Prepared By:

American Institutes for Research^®

1000 Thomas Jefferson St., NW

Washington, DC 20007

Prepared for:

U.S. Department of Education

Policy and Program Studies Service

Office of Planning, Evaluation, and Policy Development

Introduction and overview

The U.S. Department of Education (ED) is conducting an evaluation of the Carol M. White Physical Education Program (PEP). The American Institutes for Research (AIR) is under contract with ED to conduct the evaluation. The purpose of this evaluation is to describe the key design features of PEP grantees’ projects and their implementation. The study will survey project administrators of PEP grants. This OMB request is for the clearance for the survey instruments.

This document contains a description of the evaluation of the PEP and provides context on the data collection instruments for which we are seeking clearance. Parts A and B of the supporting statement for the Paperwork Reduction Act Submission and the instruments for which we are seeking clearance are contained in separate files.

Background

In establishing the PEP, Congress acknowledged the critical need to improve physical education programs for K-12 students, in order to help them make progress toward meeting state standards for physical education. Since its inception in 2001, as a major federal funding source for physical education, the PEP has supported a variety of projects that encourage fitness and healthy lifestyle choices among K-12 students. Recently, in response to the current Administration’s call for action to prevent and decrease childhood overweight and obesity, the PEP underwent substantive changes to strengthen and enhance the program’s principal objectives so that they align more closely with current best practices and research related to improving children’s health and fitness. Because of the recent changes and enhancement, it is important to document how grantees are implementing the new PEP and assess the extent to which the new program priorities and requirements produce desired student outcomes as intended.

Overview of Study Design

ED has put forth three sets of evaluation questions that pertain to basic program implementation, partnerships, data use, and student outcomes, respectively.

• Evaluation questions about basic implementation

1. What does the planning, partnership, and implementation process look like during the life cycle of a grant?

2. How do grantees interpret the elements (of the six) they choose to implement?

3. What gaps are identified by the grantees based on the School Health Index (SHI) needs assessment? Do grantees put in place specific plans to address the identified gaps?

4. To what degree do grantees change school physical activity and food policies?

5. For grantees that use grant-related funds to improve their physical education or nutrition instruction curricula, how do they use CDC’s Physical Education Curriculum Analysis Tool (PECAT) and Health Education Curriculum Analysis Tool (HECAT) to inform curricula changes?

6. How do local wellness policies influence the work of the grantees? If a grantee does not have a local wellness policy, does it adopt one?

7. Do grantees encounter unforeseen problems when implementing the new requirements?

8. What lessons are learned by the grantees that could inform future ED policy regarding the PEP program?

9. What are grantees’ priorities for funding? What are PEP funds spent on? What is cost per unit (e.g., school, student served).

• Evaluation questions about partnerships

1. How many grantees develop partnerships with supporting community entities?

2. What is the average number of partners for a grantee?

3. How do grantees work with partners?

4. What roles do partners play?

5. How do the activities planned by grantees with partnerships differ from grants without partnerships?

• Evaluation questions about data use

1. How many grantees integrate BMI in their program?

2. How many grantees inform parents about the result of their child’s BMI assessment?

3. How do grantees use data, such as BMI and other performance measures, for improvement purposes?

To answer ED’s evaluation questions, AIR proposes to survey administrators at all PEP projects. AIR will survey the project administrators at all 2010 grantees in order to gather rich information about the design and implementation of the PEP program. Two waves of PDI surveys will be administered to both local education agency (LEA) grantees and community-based organization (CBO) grantees. The Year 1 PDI survey will gather information about the design and implementation of the new awards under the revised program. The Year 3 PDI survey will focus on grantees’ reflections on accomplishment of program goals, challenges in implementing the program, and lessons learned. Together, the Year 1 and Year 3 PDI surveys will collect detailed information about the planning, operations, activities, challenges, and perceived outcomes of the PEP program as implemented by the FY 2010 cohort of grantees. Such information will provide valuable feedback to ED on the new PEP program and inform future improvement of the program.

Data Collection and Instruments

Data gathered as part of the study will serve as documentation of the implementation process of PEP projects. All instruments developed by AIR have been thoroughly reviewed by the evaluation’s technical work group (TWG) for content and suitability.

AIR will administer the two waves of PDI surveys to all of the 2010 PEP grantees. The first wave of the surveys will be administered in Summer 2011 (Year 1) and the second wave in Winter 2012/2013 (Year 3). The Year 1 PDI survey will gather information about the design and implementation of the revised grant program, such as areas of focus, number of partnerships, and usage of BMI data. The Year 3 PDI survey will focus on grantees’ reflections on accomplishment of program goals, challenges encountered in implementing the program, lessons learned, and sustainability of program activities post grant funding.

For each wave of the PDI survey, two forms have been developed: one for LEA grantees and the other for CBO grantees. That is, four PDI surveys will be administered. The surveys will be completed by key personnel (e.g., program directors) of each PEP grantee. The surveys will include both close-ended and open-ended questions in order to allow respondents the opportunity to elaborate on their responses. Both survey forms will be available on-line and incorporate skip patterns that are based on the characteristics of grantees (e. g., with partner or not, or used BMI or not).

Together, the Year 1 and Year 3 PDI surveys address ED’s questions with regard to basic implementation, partnership, and data use by collecting detailed information about the projects’ planning, operations, activities, challenges, and perceived outcomes. Through these two surveys (each with two forms: LEA and CBO), we will be able to assess whether the program as implemented appropriately reflects the new direction of the PEP and how the new program priorities and requirements are addressed by the grantees. Such information will provide valuable feedback to ED on the new PEP program and inform future improvement of the program. Therefore, we request clearance for the four PDI surveys.

Data Analyses

First, AIR will analyze the years 1 and 3 program design and implementation data to understand how grantees with different design features operate at early and late stages of the program respectively. In addition, we will combine and analyze the two waves of PDI survey data longitudinally, to document continuity and change in program implementation over time. We will conduct descriptive analysis to obtain sums, means, percentages, and frequencies of relevant program variables. Particularly, we will also inspect systematic variations based on key features of PEP projects (e.g., types of grantees, inclusion of agreement partners, grade levels served, and student demographics).

Summary

The above discussion outlines AIR’s research design for evaluating the Carol M. White PEP program. To address evaluation questions related to basic implementation, partnerships, data use, and student outcomes, AIR has proposed a to use PDI surveys to provide rich information on key design and implementation features of the program. Findings from this study will provide valuable feedback to both ED and grantees regarding the implementation of the PEP, and will inform future improvements to the program.

Notice of Trademark: “AMERICAN INSTITUTES FOR RESEARCH” and “AIR” are registered trademarks. All other brand, product, or company names are trademarks or registered trademarks of their respective owners.

1000 Thomas Jefferson Street, NW | Washington, DC 20007‑3835