Responses to OMB Comments

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138-1168

Responses to Questions on OMB Forms Clearance Package

Reading First Implementation Study:

2008-2009

1. How does ED reconcile two sets of recent findings (one from IES and one from an AIR study) that appear to indicate opposite findings about whether Reading First improves reading proficiency?

The findings reported in the IES report are from a quasi-experimental study that compared outcomes for schools receiving Reading First funding to outcomes for a comparable set of schools that did not receive Reading First funding. In both sets of schools, students were administered an independent assessment. The purpose of that study was to assess the impact of receipt of Reading First funding on classroom instruction and on student reading performance. The second report, released by OESE, reports state-reported performance levels for Reading First schools and district grantees only; there are no data reported for comparison schools or districts. The purpose of this report is to provide annual information about program-wide performance.

2. How does the currently proposed data collection relate to each of these two previous efforts?

The current data collection is distinct from the two data collection activities described above. The proposed data collection is designed to address different research questions than those posed by those prior studies. The IES study is a national impact study designed to assess the average impact of the program on key instructional and student achievement outcomes. The AIR study was designed to present state-specific information on student reading performance on state-selected measures. Neither of these two efforts conducted interviews with state directors, which is the primary data collection activity for the current study.

3. Why is it critical to survey states prior to implementing budget cuts? Hypothetically, if states plan to make changes that they ultimately do not make (learned per the planned 2009 data collection), what is the utility of knowing this information? Does the program office plan to intervene or offer guidance if it does not believe that the plans are appropriate?

Information about patterns in states’ responses to reduced funding will inform ED’s subsequent efforts to provide technical assistance that is responsive to grantees’ needs.

4. Does ED plan to publish state specific results? This is not clear in the Supporting Statement.

ED does not plan to publish state-specific information.

5. If not, why does ED require a universe (versus sample) collection?

ED requires a universe data collection because the overall population of state directors is small (n=53, including territories). To estimate the population percentage of some characteristic of interest (for example, the percentage of directors who report that they are continuing to provide professional development to teachers) with a margin of error of plus or minus 5 percentage points at 95% confidence level, one would require 44 states. In addition, ED plans to report on subgroups of states, using such stratifiers as region (west, south, central, northeast) and population/school enrollment (high, medium, low). The marginal increase in the cost of collecting data from 44 to 53 states is small relative to the benefit of having complete data from all states that will allow such stratification. For these reasons, ED is planning to include all states in the proposed data collection activity.

6. Is ED intending to offer confidentiality to respondents? If so, under what legal authority are these data protected? Please note that we do not believe it is possible to offer confidentiality and produce state level findings at the same time.

ED is not intending to offer confidentiality to respondents. However, responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific individual or state. ED will not provide information that identifies an individual state to anyone outside the study team.

7. To what surveys specifically is ED referring in A-4 of the Supporting Statement?

The reference to surveys in section A-4 is not accurate. It has been rewritten and resubmitted to you.

8. Has ED used the full introductory script proposed for the telephone interview previously? Has ED done any pre-testing of the script? We note that it is much longer that most we review.

ED has used similar introductory scripts for other telephone interviews. Additionally, ED will pre-test this specific script in mid-August 2008, before beginning the full data collection activity.

Responses to Questions on OMB Forms Clearance Package 2