12-10-07 memo

Responses to OMB Questions/Requests of December 4, 2007

OMB questions/requests are indicated in bold italics below.

A. Please send revised Part A and B of the supporting statement

Response to Comment A:

Attached please find the revised supporting statement (Part A and B).

B. Regarding response to Q 2: As both treatment and control teachers are receiving professional development free of charge, they should not be further incentivized. Please eliminate references to gift cards.

Response to Comment B:

We eliminated references to gift cards. The changes are reflected in the revised supporting statement.

C. Regarding response to Q3: The REL describes a burden of at least a couple of hours on administrative staff that should be added to burden estimates for the study.

Response to Comment C:

The additional burden hours were included in the revised burden estimates which were submitted to OMB last month as part of our response. These additional hours are also noted in the revised supporting statement.

D. Is “free/reduced price lunch” the only student level variable that you specifically anticipate difficulty in obtaining? If not, what are the others? If there are several substantive student-level variable missing, why would substituting school-level data into the student level of the model be more appropriate than reducing the model to two levels?

This question is in response to our previous response to Q4. The original question by OMB and PREL’s response is reproduced below in italics for your information.

Original OMB Question #4. Why do you anticipate difficulty in collecting complete student level variables? How important are these variables to your modeling?

Original PREL Response to Question #4: We do not anticipate any unusual difficulty in collecting complete student-level variables. While there may be some data collection challenges that are specific to this study with regard to student-level data, the level of risk of such challenges is what we would expect in any data collection process for this type of information.

Not all student-level information may be readily or uniformly available in all entities, especially in American Samoa or CNMI. For example, some student data—such as free/reduced lunch status or ELL status—may not be available; if available, these data may not be consistently recorded. The availability and reliability of such student-level information will be closely checked. We expect basic student-level data, such as students’ grade level and gender, will be available, and we will use those uniformly available data as baseline covariates in estimating the full-sample effects. When a student-level variable is not available, we will seek classroom-level or school-level information.

Response to Comment D:

We would like to clarify that the reference to free/reduced lunch status was used to illustrate the type of student data we would like to collect. We are not inferring that any particular variable is likely to be missing or inconsistent. We apologize for any confusion caused by referencing these specific variables. We simply intended to state that in a hypothetical case, if some student data are missing or inconsistent, we would have plans to deal with such a situation. As stated in our response, we do not anticipate any unusual difficulty in collecting complete student-level variables. We do not anticipate that substantive student-level variables will be missing.

Specifically, our approach to the use of student-level baseline covariates is as follows:

1. Regardless of the availability of specific student-level baseline covariates, our estimation models will include school-level covariates. This preserves our ability to minimize random variation in background characteristics between schools (our unit of randomization). We will use estimation methods to control for clustering (such as hierarchical linear modeling) and will utilize all levels of information available.

2. If data on specific variables are available for most students, but not all, we will impute missing covariates for the missing students. This enables those students to remain in the analysis and allows us to conduct the analysis at the student level, thus preserving valuable information on variation in outcomes and background variables across students.

3. If data on specific variables are not available at the individual student level for all students, we will still include other available student-level data as well as school-level data as covariates in the same model (for estimating a student-level outcome).