HSIS Tracking OMB Supporting Statement A FINAL 033012

The Office Planning, Research and Evaluation of the Administration for Children and Families (ACF), U.S. Department of Health and Human Services (DHHS), is submitting this Request for OMB Review in support of an ongoing project to collect contact information from children and families who participated in the Head Start Impact Study (HSIS). In anticipation of conducting an HSIS follow-up during or soon after the high school years and to ensure high response rates for this follow-up, ACF has awarded a contract to Westat to continue to contact and track families for five years through the end of high school with information to be collected in spring 2012, 2013, 2014, 2015 and 2016 for Tracking Head Start Impact Study Participants Beyond 8^th Grade. Currently, we are seeking OMB approval for the first three years’ data collection from spring 2012 through spring 2014. Data collection will include telephone parent tracking interviews (and in-person contact when necessary) to maintain updated contact information necessary for a follow-up study. Such a study would be beneficial for the examination of the long-term impacts of Head Start.

OMB has previously approved packages for these prior studies:

The Head Start Impact Study (HSIS) – OMB approved the initial package for HSIS in September 2002 to randomly assign 4,667 children to a Head Start group or control group. Data was collected on these children and their families through the child’s first grade year in spring 2006 (OMB # 0970-0229, Expiration Date: 09/30/2005 and the HSIS continuation OMB # 0970-0229, Expiration Date: 07/30/2006).
The Third Grade Follow-up to the Head Start Impact Study – OMB approved another package, under the same OMB number in July 2006 for this follow-up with the children and families through the children’s third grade year in spring 2008 (OMB # 0970-0229, Expiration Date: 07/30/2008).
The Tracking of Former Participants in the Head Start Impact Study – OMB approved this most recent package under the same OMB number in February 2009 to continue tracking the same children and families each spring for three years through spring 2011 (OMB # 0970-0229, Expiration Date: 02/29/2012).

The current project for which we are seeking OMB approval for the first three years’ data collection from spring 2012 through spring 2014 is:

The Tracking Head Start Impact Study Participants Beyond 8^th Grade -- The previous HSIS studies, listed above, have been highly successful in recruiting and maintaining participants over time, achieving nearly an 80 percent response rate in all waves of the study. The Tracking Head Start Impact Study Participants Beyond 8^th Grade is designed to continue tracking to maintain these high response rates during years in which no large-scale follow-up study will be conducted. We will continue to conduct telephone tracking interviews with in person follow-up as necessary to update the respondent's location and contact information. This information will be collected from parents or guardians in the same manner as was implemented for the Tracking of Former Participants in the Head Start Impact Study.

Background

Overview of Head Start. Head Start has served over 27 million preschool children and their families since it began in 1965 as a six-week summer program for children of low-income families. The program provides comprehensive early child development services to low-income children, their families, and communities. Head Start has evolved over time to include a wide variety of program options based on the specific situations and resources of the communities to meet the changing needs of the children and families it serves. In addition, many programs are now partnering with non-Head Start agencies and/or combining funds from various sources to coordinate services that best address the needs of children and families.

As Head Start’s Federal appropriation has grown, ($96 million in summer 1965 to $7.6 billion for FY 2011) initiatives have called for improved outcomes and accountability for all federal programs (e.g., Chief Financial Officers Act, Government Performance and Results Act of 1993). During the rapid expansion of Head Start, the U.S. General Accounting Office (GAO) released two reports underlining the lack of rigorous research on Head Start’s effectiveness noting that “…the body of research on current Head Start is insufficient to draw conclusions about the impact of the national program" (GAO, 1997). The 1998 report added, “…the Federal government’s significant financial investment in the Head Start program, including plans to increase the number of children served and enhance the quality of the program, warrants definitive research studies, even though they may be costly” (GAO, 1998).

Overview of Previously Conducted Head Start Impact Studies. Based upon GAO recommendation, and the testimony of research methodologists and early childhood experts, Congress mandated in Head Start’s 1998 reauthorization that DHHS conduct research to determine, on a national level, the impact of Head Start on the children it serves. Congress called for an expert panel to develop recommendations regarding the study design to “…determine if, overall, the Head Start programs have impacts consistent with their primary goal of increasing the social competence of children, by increasing the everyday effectiveness of the children in dealing with their present environments and future responsibilities, and increasing their school readiness” (42 USC 9801, et.seq.). Congress also declared that the research should also consider variables such as whether Head Start strengthens families as the nurturers of their children and increases children’s access to other education, health, nutritional, and community services.

To design such a study, the Department convened a committee of distinguished experts, the Advisory Committee on Head Start Research and Evaluation, that considered the major issues and challenges in designing a rigorous research study that is both credible and feasible, and the committee recommended a framework for the design of the Head Start Impact Study (HSIS). A contract was awarded in October 2000 to Westat, in collaboration with the Urban Institute, American Institutes for Research, and Decision Information Resources to conduct the Head Start Impact Study as mandated by the Coats Human Services Amendments of 1998, PL 105-285. Following HSIS, Westat and its subcontractors were awarded the Third Grade Follow-up to the Head Start Impact Study to study the children and families through spring 2008 and the children’s third grade year. Subsequently, Westat was awarded the Tracking of Former Participants in the Head Start Impact Study to continue to follow the families through spring 2011.

Following is a brief description of each of these past studies:

The Head Start Impact Study (HSIS) original design called for collecting comparable data on two cohorts of newly entering children (a three-year old cohort and a four-year old cohort) and their families who were randomly assigned to either a treatment group (enrolled in Head Start) or a control group (that were not enrolled in Head Start, but were permitted to enroll in other available services in their community selected by their parents or be cared for at home). To draw the national sample, all eligible grantees/delegate agencies were clustered geographically with a minimum number of eight grantees/delegate agencies within each cluster. The clusters were grouped into 25 strata based on state pre-K and childcare policy,^¹ race/ethnicity of the Head Start children served, urban/rural status, and region. Next, one cluster with probability proportional to the total enrollment of three- and four-year olds in the cluster, was selected from each stratum and approximately three grantee/delegate agencies were randomly selected from each cluster. From each of the 75 randomly selected grantees/delegate agencies, approximately 48 children per grantee/delegate agency were assigned to the Head Start treatment group and about 32 children were assigned to the control group. Sample children could not have been previously enrolled in Head Start. To avoid a sample size shortfall, small centers on the frame were grouped together within a program to form center groups, each center group with a combined reported first year enrollment of at least 27 children. The selection and random assignment of the sample comprising 4,667 children occurred during the spring/summer of 2002. These children were followed from Head Start enrollment through the end of their first grade year. The distribution of children by cohort or age group and by status (treatment or control group) is presented in Table 1.

Table 1. Number of Children in the Head Start and Non-Head Start Groups by Age Cohort

Age Cohort	Head Start (Treatment Group)	Non-Head Start (Control Group)	Total Sample
Three-year olds	1,530	1,029	2,559
Four-year olds	1,253	855	2,108
Total	2,783	1,884	4,667

The Third Grade Follow-Up to the Head Start Impact Study was based largely on work that had already been completed for the HSIS, and provided an opportunity to examine the degree to which the impacts of Head Start on initial school readiness are changed by children’s third grade school experiences and the family/community factors associated with the child during the school years. This study provided the means to broaden the scope of analysis to include factors that had not yet been examined as part of the HSIS. The Third Grade Follow-up data collection included parent interviews, child assessments and child self-description questionnaires, teacher and principal surveys, and teacher ratings of children. The Third Grade Follow-up continued to collect HSIS comparable data on the entire sample of 4,667 children and families through the children’s third grade year.

The Tracking of Former Participants in the Head Start Impact Study was designed to track the families, maintain current contact information in anticipation of ACF conducting a follow-up study, and to ensure adequate response rates for that follow-up study. Data collection included parent tracking interviews. The Tracking study continued tracking 4, 243 children and their families from the original sample in spring 2009, 2010 and 2011, excluding, with ACF agreement, 230 hard refusals (those who adamantly refused and/or threatened if we continued to contact them) and 194 families that never participated during HSIS or the Third Grade Follow-up.

Following is a description of the current study for which we are seeking OMB approval:

The Tracking Head Start Impact Study Participants Beyond 8^th Grade will follow 4,235 children and their families from the original sample, excluding those who were excluded from prior rounds and an additional 8 families who were designated hard refusals between spring 2008 and spring 2011 during the Tracking of Former Participants in the Head Start Impact Study. This effort is designed to maintain these high response rates during years in which no large-scale follow-up study will be conducted. To maintain adequate sample size, telephone interviews will be conducted to update the respondent's location and contact information. Data collection under this contract is planned to occur in the spring of 2012, 2013, 2014, 2015 and 2016, and we are currently seeking OMB approval for spring 2012 through spring 2014. A parent tracking interview form will be used to verify and update, as necessary, families’ contact information and the child’s school information. The tracking interviews will primarily be conducted over the telephone with in-person follow-up as necessary. Tracking interviews will take about 20 minutes to complete. The Parent Tracking Interview is provided in Appendix A.

Research Questions

For this tracking study, as for thesimilar one that preceded it, there will be no research questions, nor any analyses to examine any research questions. The information being collected is solely for the purpose of ensuring that future data collection efforts will be successful in reaching study participants.

A.2 Purpose of Survey and Data Collection Procedures

The primary purpose of this follow-up is to continue tracking the children and families to ensure high response rates in anticipation of follow-up data collections. This follow-up will be beneficial for the examination of long-term impacts of Head Start. The HSIS studies (HSIS, Third Grade Follow-up and Tracking described in Section A.1) have been highly successful in recruiting and maintaining participants over time, achieving nearly an 80 percent response rate in all waves. The current project will build upon these prior studies:

Many challenges are presented by a longitudinal study with a national sample. Over time, families move and become more difficult to locate. We have been successful in gaining high cooperation from respondents for all prior rounds of data collection and tracking. High response rates were achieved for the parent interviews and the child assessments as presented in Table 2.

Table 2. Percent of Parent Interviews and Child Assessments Complete by Data Collection Period

Data Collection Activity	Response Rate (percent with complete or mostly complete instrument)
Data Collection Activity	Parent	Child
Fall 2002	86%	80%
Spring 2003	83%	84%
Tracking Fall 2003	84%	--
Spring 2004	81%	81%
Tracking Fall 2004	83%	--
Spring 2005	81%	78%
Tracking Fall 2005	83%	--
Spring 2006	80%	77%
Tracking Spring 2006 (4-year-old cohort)	82%	--
Tracking Fall 2006	81%	--
Tracking Spring 2007 (3-year-old cohort)	81%	--
Spring 2007	77%	72%
Tracking Fall 2007	81%	--
Tracking Spring 2008 (4-year-old cohort)	78%	--
Spring 2008	78%	74%
Tracking Spring 2009	86%	--
Tracking Spring 2010	86%	--
Tracking 2011	85%	--

NOTE: N=4,667through Spring 2008, N= 4243 from Spring 2009 through Spring 20 11.

A.3 Improved Information Technology to Reduce Burden

The tracking interviews will be conducted primarily over the telephone with in-person interviews as necessary.

A.4 Efforts to Identify Duplication

In the late 1990’s, the US General Accounting Office (GAO) released two reports concluding that (1) “…the Federal government’s significant financial investment in the Head Start program, including plans to increase the number of children served and enhance the quality of the program, warrants definitive research studies, even though they can be costly”(GAO, 1998) and (2) this information need could not be met because “…the body of research on current Head Start is insufficient to draw conclusions about the impact of the national program” (GAO, 1997).

The Head Start Impact Study was the first – and continues to be the only – large-scale randomized experiment testing the effectiveness of Head Start. One purpose of the Head Start Impact Study was to measure the impact of Head Start on children’s early development and school readiness. The Third Grade Follow-up provided the opportunity to look at longer term effects later in elementary school. This continued tracking will ensure sufficient maintenance of the original Head Start Impact Study sample to allow a longer-term follow-up during or soon after high school.

A.5 Involvement of Small Organizations

No small businesses or other small entities will be involved in the data collection for this tracking study

A.6 Consequences of Less Frequent Data Collection

As recommended by the Government Accounting Office and mandated by Congress, “definitive research studies” are legislatively required to assess the effectiveness of Head Start nationally on the school readiness of participating children. Despite increasing expenditures, including an appropriation of $7.6 billion in fiscal year 2011, in 1997 GAO indicated that “…the body of research on current Head Start is insufficient to draw conclusions about the impact of the national program” (GAO, 1997). The Head Start Impact Study was the first – and continues to be the only – large-scale randomized experiment testing the effectiveness of Head Start. The follow-ups conducted with this sample in third grade and potentially in adulthood provide the most rigorous estimate of the long-term effects of the Head Start program. The Tracking of Former Participants in the Head Start Impact Study produced high response rates and provided updated contact information annually from spring 2009 through spring 2011. The current project, Tracking Head Start Impact Study Participants Beyond 8^th Grade, will help to ensure high response rates in a follow-up study should ACF decide to implement an evaluation following the high school years. Longer periods between data collections would risk lower response rates with respondent interest possibly waning and family movement would be more difficult to track.

A.7 Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A.8 Federal Register Notice and Consultation

The public announcement for the Continued Tracking of Families in the Head Start Impact Study was published in the Federal Register on May 6, 2011 (Vol. 76, No. 88, p 26303). The text of the announcement is contained in Appendix B. No comments were received from the public through this notice.

Information concerning the Head Start Impact Study was included in the “Report of the Advisory Committee on Head Start Research and Evaluation” required as part of the Head Start Amendments of 1998. Consultation meetings were held with the Advisory Committee on Head Start Research and Evaluation on January 12, 2001, June 16-17, 2003, and September 28-29, 2005. In addition, a consultation meeting was held with the Advisory Committee on Head Start Accountability and Educational Performance Measures on June 16, 2005. The Advisory Committee on Head Start Research and Evaluation met on January 25-26, 2011 to discuss the HSIS study design, findings, and plans for future work. Committee members suggested that a low-cost follow-up study was important and during or soon after high school may be the best time for conducting such a study. (Advisory Committee Members are listed in Appendix C.)

Given that this data collection involves only tracking current participants, no additional consultation was conducted.

A.9 Payment of Respondents

In order to minimize the burden placed on families for participating in the tracking study and to maximize response rates, we will provide parents with an incentive ($20) for each completed tracking interview. Parents have received $20 since 2003 for tracking in the Head Start Impact Study, the Third Grade Follow-up and the Tracking of Former Participants in the Head Start Impact Study. We are currently seeking OMB approval for three additional annual data collections from spring 2012 through spring 2014 and anticipate the likelihood of needing to maintain cooperation for this study over two more rounds in spring 2015 and 2016. It is important to continue a cash incentive to thank parents for their participation and maintain consistent data collection procedures. Continuing and maintaining this amount over the course of the project will aid in maintaining parent interest, cooperation, and participation.

Recent research indicates positive effects for the use of cash incentives. In an experiment, respondents at one wave of a longitudinal study were randomly assigned to receive the same $20 incentive they had received in earlier waves or a $30 incentive or a $50 incentive. “Those offered $50 had a higher response rate than those offered $20, and this positive impact persisted for at least the next four waves of biennial data collections” (Rodgers, 2011). In another study, researchers found positive effects for those who had earlier refused, “Providing incentives to respondents, who previously refused to participate in the last survey round, significantly boosted response rates, and resulted in longer interviews and more items answered” (Zagorsky and Rhoton, 2008). In another study, researchers looked at the use of incentives to reduce nonresponse in random digit dial telephone surveys and reported that for extended interviews (as opposed to screeners), “A number of studies have found that promised incentives of $15 – $35 increase response rates” (Cantor, O’Hare and O’Connor, 2008). In a recent review of the use of incentives in longitudinal surveys, Laurie and Lynn (2008) concluded, “Overall, it seems clear that the use of respondent incentives is an important element of the strategy to minimize attrition for many longitudinal surveys. The evidence suggests consistently that attrition rates would be higher in the absence of incentives”.

A.10 Confidentiality of Respondents

All Westat staff members sign the Westat pledge of confidentiality for the study. In addition, field staff members sign a Westat confidentiality pledge. The two confidentiality pledges are included in Appendix D.

For some parent respondents, the issue of privacy of information, particularly relating to address and telephone information collected for later tracing of respondents, is a matter of great concern. Participants will be assured that the information collected will be kept private to the extent permitted by law and will be used for research purposes only by the research team, and that contact name and address information and other survey data will not be given to bill collectors, legal officials, other family members, etc.

We will implement procedural steps, similar to the steps previously used in the HSIS, the Third Grade Follow-up to the HSIS, and the Tracking of Former Participants in HSIS, to increase respondent confidence in our privacy procedures. We will generate a set of identification labels with a unique respondent ID number and bar code. These labels will be affixed to each of the data collection instruments for a respondent. The tracking interview also will have a label on the front page with the respondent’s name. The respondent’s name will be on a peel-off label and the label will be removed and destroyed prior to sending the completed tracking interview to Westat. The use of bar codes in conjunction with the numbered identification labels enables the receipt control staff to enter cases by reading the bar code with a wand, making receipt of completed interview packages also more efficient.

A.11 Sensitive Questions

We do not anticipate that any of the questions asked will be of a sensitive nature. The purpose of the interview and how the data will be used will be explained to all participants. Participants will be reassured in person and in writing that their participation in the study is completely voluntary. A decision not to participate will not affect their standing in any government program, and if they choose to participate, they may refuse to answer any question they find intrusive. All individuals’ responses will be kept private and none of their answers will be reported to any program, agency, or school but will be combined with the responses of others so that individuals cannot be identified. All interviews will take place in a setting where the respondent's privacy can be assured.

A.12 Estimation of Information Collection Burden

Table 3 presents data on the annual burden for respondents to the new tracking study for each data collection point. The burden estimates will be the same for the three years’ data collections from spring 2012 through spring 2014 for which we are currently requesting OMB approval. Hence the total number of responses across these three years is 12,705 responses and 4,236 hours are the total hours requested across the three years’ data collections.

Table 3. Estimated Annual Response Burden for Respondents in the Tracking Head Start Impact Study Participants Beyond 8^th Grade

Instrument	Annual Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Total Burden Hours	Average Hourly Wage	Total Annual Cost
Parent Tracking Interview	4, 235	1	1/3	1,412
Estimated Annual Burden Sub-total				1,412

Total Respondents for each Year: 4,235

Total Responses for each Year: 4,235

Total Burden Hours for each Year: 1,412

A.13 Cost Burden to Respondents or Record Keepers

There are no direct monetary costs to participants other than their time to participate in the study.

A.14 Estimate of Cost to the Federal Government

The total cost to the Federal Government for this tracking study is estimated to be $2.387 million over the first three years of the contract. These costs include development of the tracking materials, data collection, and preparation of a data file. Respondent expenses and incentives are included in these costs. The average annualized cost over the first three years of the contract is estimated to be approximately $795,720 per year.

A.15 Changes in Burden

There are no planned program changes. OMB will be notified promptly if any changes are recommended by outside consultants or identified by project staff. The estimated approved annual burden for the prior study, “Tracking of Former Participants in the Head Start Impact Study”, which was approved by OMB on February 11, 2009, was 4,667 respondents for ¼ hour or 1,167 hours. The estimated burden for Tracking Head Start Impact Study Participants Beyond 8^th Grade is 4, 235 respondents for 1/3 hour or 1,412 hours.

A.16 Plan and Time Schedule for Information Collection, Tabulation and Publication

Since this tracking study is not designed to answer any research questions, there are no plans for statistical analysis of these data. A single indicator, the percentage of families successfully tracked, will be created to indicate the success of the overall tracking effort to advise the study for future data collection. No publications will be created from this tracking study.

Currently, we are requesting OMB approval for the first three data collections for this tracking study to take place in spring 2012, 2013, and 2014, (and in future, we anticipate seeking OMB approval for two additional rounds in spring 2015 and spring 2016) for all families. Data collections are planned for mid-March through mid-May, the same months as data collection for the Tracking of Former Participants in the Head Start Impact Study in spring 2009, 2010 and 2011.

A.17 Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

Bibliography

Cantor, D., O’Hare, B. and O’Connor, K. (2007) “The Use of Monetary Incentives to Reduce Non-Response in Random Digit Dial Telephone Surveys” pp. 471-498 in J. M. Lepkowski, C. Tucker, J. M. Brick De Leeuw, E., Japec, L., Lavrakas, P. J., Link, M. W., & Sangster, R. L. (Eds.), Advances In Telephone Survey Methodology, New York: J.W. Wiley and Sons, Inc.

Laurie, H. and P. Lynn (2009) “The Use of Respondent Incentives on Longitudinal Surveys.” pp. 205 – 231, in P. Lynn (ed) Methodology of Longitudinal Surveys” John Wiley & Sons: Southern Gate.

Rodgers, Willard L. (2011) “Effects of Increasing the Incentive Size in a Longitudinal Study.” Journal of Official Statistics, Vol. 27, No. 2, 2011, pp. 279-299.

Schulman, K., Blank, H., and Ewen, D. (1999). Seeds of success: State prekindergarten initiatives 1998-1999. Washington, DC: Children’s Defense Fund.

U.S. General Accounting Office. (1997). Head Start: Research provides little information on impact of current program. Washington, DC: U.S. Government Printing Office.

U.S. General Accounting Office. (1998). Head Start: Challenges in monitoring program quality and demonstrating results. Washington, DC: U.S. Government Printing Office.

Zagorsky, Jay and Rhoton, Patricia (2008), “The Effects of Promised Monetary Incentives on Attrition in a Long-Term Panel Survey” (Fall 2008). Public Opinion Quarterly, Vol. 72, Issue 3, pp. 502-513.

1 Data to define these strata were obtained from Children’s Defense Fund (1999), Seeds of Success report.

File Type	application/msword
Author	Freeland_s
Last Modified By	CTAC
File Modified	2012-04-19
File Created	2012-04-19

HSIS Tracking OMB Supporting Statement A FINAL 033012