Evaluation of Child Care Subsidy Strategies

memorandum

Social and Economic Policy

Date	January 25, 2008
To	Ivelisse Martinez-Beck, Brendan Kelly, Naomi Goldstein
From	Ann Collins
Subject	Request for Expedited Review of Revisions to package for Evaluation of Child Care Subsidy Strategies: Massachusetts, Illinois, and Washington (OMB NO.: 0970-030)

As you know, we have submitted a request to change two aspects of our data collection for the Child Care Subsidy Evaluation (Package Number referenced above). They were to do the following:

• Substitute one subtest of a standardized child assessment measure (the PLS-4) with a subtest of another standardized child assessment measure (the PLS-4). (No change in burden.)

• Increase the incentive amount for the Illinois study from $20 to $50 for completed interviews for from study participants to increase response rate. (No change in burden).

Both of these data collections are happening at the end of the intervention period.

Massachusetts

The Massachusetts study tests the effects of LearningGames, a developmental curriculum, in family child care homes using a random assignment design. The original plan called for using the PLS-4 as the tool for evaluating impacts of LearningGames on child development for all children, regardless of age. As you know, we requested the change in assessment substituting one subtest of the PLS-4 with the Bracken because of the experiences of another project funded through a grant from the Child Care Bureau. We wanted our data collection measures to conform with those used by this grant-funded project in order to maximize the information available for policymakers about a range of potential interventions in family child care homes. (We also learned that the other project found that the subtest of the PLS-4 that we had originally planned to use was not sensitive enough to detect child outcomes and therefore was not worth doing.)

Timeline. For the Massachusetts experiment, the intervention period ended in December 2007 and therefore we are awaiting feedback about whether we will be able to substitute these measures before doing our final data collection. We will need to complete the final data collection for Massachusetts by the end of February 2008. Therefore, if we do receive clearance to substitute these measures by February 8, 2008, we will be forced to omit the Bracken assessment. This will greatly hamper our ability to detect child outcomes and will limit the usefulness of the study and its comparability to other research on the effects of quality improvements in family child care homes.

Illinois

As described to OMB in 2005, approximately 1,900 parents in Cook County, Illinois, who applied for child care subsidies with income between 50 and 65 percent of state median income (SMI) were asked to participate in a random assignment study. Without the study, these families would not have been eligible for child care subsidies because their income placed them above state income guidelines. Half of the families who agreed to be in the study were approved to receive subsidies for two years.

In Illinois, we requested a change to a higher incentive amount to decrease non-response bias. Our experience indicates to us that the higher incentive amount will increase the number of respondents. The original data collection plan included a $20 incentive for study participants who completed a 35-minute telephone interview. We now have requested a $50 incentive for the remaining participants who have not yet completed the interview. Our experience indicates that the increased incentive will greatly improve the quality of the data collection. In the study design, the survey is the only source of information about how the use of child care subsidies affects the parents’ selection of and satisfaction with child care, as well as its stability, as such information is not available through administrative records, which is the other source for the study. The quality of this information will be much enhanced if we can minimize non-response bias.

Timeline. As you know, Illinois study participants were recruited on a rolling basis. The two-year intervention period ends for the last wave of study participants in April 2008; four weeks later is the final date for any potential data collection as we believe any data collected beyond May 2008 will not be meaningful because it will require participants to provide too much detailed information from recall. Therefore, as each week goes by, the likelihood that the higher incentive will improve non-response bias goes down as we expect that we will receive more refusals because potential study participants have been contacted too many times. Therefore, we believe that if we do not receive approval to increase the incentive by mid-February 2008, we believe that the increased incentive amount will not have the beneficial effect.

2