OMB Supporting Statement Part A HSIS Tracking

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for an ongoing project to collect contact information from children and families who participated in the Head Start Impact Study (HSIS). In anticipation of conducting a potential HSIS follow-up during or soon after the high school years and to ensure high response rates for this follow-up, ACF has awarded a contract to Westat to continue to contact and keep in touch with families for five years through the end of high school with information to be collected in spring 2012, 2013, 2014, 2015 and 2016 for the Head Start Impact Study (HSIS) Participants Beyond 8^th Grade. We collected information in spring 2012, 2013 and 2014 as approved under OMB #0970-0229 (NOA 4/20/2012) and now seek OMB approval for the last two years’ data collection in spring 2015 and spring 2016. Data collection will include telephone parent follow-up interviews (and in-person contact when necessary) to maintain updated contact information necessary for a potential follow-up study. Additionally, the parent interview will include a small set of items on children’s special education needs, grade retention, school safety, school engagement, and parental monitoring to provide information on factors during adolescence that may influence long-term impacts of Head Start examined in a potential follow-up study.

OMB has previously approved the following information collection requests for the Head Start Impact Study:

The Head Start Impact Study (HSIS)-OMB approved the initial package for HSIS in September 2002 to randomly assign 4,667 children to a Head Start group or control group. Data was collected on these children and their families through the child’s first grade year in spring 2006 (OMB # 0970-0229, Expiration Date: 09/30/2005 and the HSIS continuation OMB # 0970-0229, Expiration Date: 07/30/2006).
The Third Grade Follow-up to the Head Start Impact Study (HSIS)-OMB approved another package, under the same OMB number in July 2006 for this follow-up with the children and families through the children’s third grade year in spring 2008 (OMB # 0970-0229, Expiration Date: 07/30/2008).
The Tracking of Former Participants in the Head Start Impact Study (HSIS)-OMB approved this package under the same OMB number in February 2009 to contact and keep in touch with the same children and families each spring for three years through spring 2011 (OMB # 0970-0229, Expiration Date: 02/29/2012).
Head Start Impact Study (HSIS) Participants Beyond 8^th Grade-OMB approved this package under the same OMB number in April 2012 to contact and keep in touch with the same children and families each spring for three years through spring 2014 (OMB # 0970-0229, Expiration Date: 04/30/2015).

We are seeking OMB approval for the last two years’ data collection in spring 2015 and spring 2016 for:

Head Start Impact Study (HSIS) Participants Beyond 8^th Grade-The previous HSIS studies, listed above, including the first three years of this project have been highly successful in recruiting and maintaining participants over time, achieving nearly an 80 percent response rate in all waves of the study. The last two years of the HSIS Participants Beyond 8^th Grade is designed to contact and keep in touch with families to maintain these high response rates during years in which no large-scale follow-up study will be conducted. We will continue to conduct telephone interviews with in-person follow-up as necessary to update the respondent's location and contact information. This information will be collected from parents or guardians in the same manner as was implemented for the first three years of the HSIS Participants Beyond 8^th Grade with slight modifications. The telephone interview will also include a small set of items on children’s special education needs, grade retention, school safety, school engagement, and parental monitoring to provide information on factors during adolescence that may influence long-term impacts of Head Start examined in a potential follow-up study.

Study Background

Overview of Head Start. Head Start has served over 31 million preschool children and their families since it began in 1965 as a six-week summer program for children of low-income families. The program provides comprehensive early child development services to low-income children, their families, and communities. Head Start has evolved over time to include a wide variety of program options based on the specific situations and resources of the communities to meet the changing needs of the children and families it serves. In addition, many programs are now partnering with non-Head Start agencies and/or combining funds from various sources to coordinate services that best address the needs of children and families.

As Head Start’s Federal appropriation has grown, ($96 million in summer 1965 to $8.6 billion for FY 2014) initiatives have called for improved outcomes and accountability for all federal programs (e.g., Chief Financial Officers Act, Government Performance and Results Act of 1993). During the rapid expansion of Head Start, the U.S. General Accounting Office (GAO) released two reports underlining the lack of rigorous research on Head Start’s effectiveness noting that “…the body of research on current Head Start is insufficient to draw conclusions about the impact of the national program" (GAO, 1997). The 1998 report added, “…the Federal government’s significant financial investment in the Head Start program, including plans to increase the number of children served and enhance the quality of the program, warrants definitive research studies, even though they may be costly” (GAO, 1998).

Overview of Previously Conducted Head Start Impact Studies. Based upon GAO recommendation, and the testimony of research methodologists and early childhood experts, Congress mandated in Head Start’s 1998 reauthorization that HHS conduct research to determine, on a national level, the impact of Head Start on the children it serves. Congress called for an expert panel to develop recommendations regarding the study design to “…determine if, overall, the Head Start programs have impacts consistent with their primary goal of increasing the social competence of children, by increasing the everyday effectiveness of the children in dealing with their present environments and future responsibilities, and increasing their school readiness” (42 USC 9801, et.seq.). Congress also declared that the research should also consider variables such as whether Head Start strengthens families as the nurturers of their children and increases children’s access to other education, health, nutritional, and community services.

To design such a study, the Department convened a committee of distinguished experts, the Advisory Committee on Head Start Research and Evaluation, that considered the major issues and challenges in designing a rigorous research study that is both credible and feasible, and the committee recommended a framework for the design of the Head Start Impact Study (HSIS). A contract was awarded in October 2000 to Westat, in collaboration with the Urban Institute, American Institutes for Research, and Decision Information Resources to conduct the HSIS as mandated by the Coats Human Services Amendments of 1998, PL 105-285. Following HSIS, Westat and its subcontractors were awarded the Third Grade Follow-up to the HSIS to study the children and families through spring 2008 and the children’s third grade year. Subsequently, Westat was awarded the Tracking of Former Participants in the HSIS to continue to follow the families through spring 2011 and the HSIS Participants Beyond 8^th Grade to follow the families through spring 2016.

Following is a brief description of each of these past studies:

The HSIS original design called for collecting comparable data on two cohorts of newly entering children (a three-year old cohort and a four-year old cohort) and their families who were randomly assigned to either a treatment group (enrolled in Head Start) or a control group (that were not enrolled in Head Start, but were permitted to enroll in other available services in their community selected by their parents or be cared for at home). To draw the national sample, all eligible grantees/delegate agencies were clustered geographically with a minimum number of eight grantees/delegate agencies within each cluster. The clusters were grouped into 25 strata based on state pre-K and childcare policy,^¹ race/ethnicity of the Head Start children served, urban/rural status, and region. Next, one cluster with probability proportional to the total enrollment of three- and four-year olds in the cluster, was selected from each stratum and approximately three grantee/delegate agencies were randomly selected from each cluster. From each of the 75 randomly selected grantees/delegate agencies, approximately 48 children per grantee/delegate agency were assigned to the Head Start treatment group and about 32 children were assigned to the control group. Sample children could not have been previously enrolled in Head Start. To avoid a sample size shortfall, small centers on the frame were grouped together within a program to form center groups, each center group with a combined reported first year enrollment of at least 27 children. The selection and random assignment of the sample comprising 4,667 children occurred during the spring/summer of 2002. These children were followed from Head Start enrollment through the end of their first grade year. The distribution of children by cohort or age group and by status (treatment or control group) is presented in Table 1.

Table 1. Number of Children in the Head Start and Non-Head Start Groups by Age Cohort

Age Cohort	Head Start (Treatment Group)	Non-Head Start (Control Group)	Total Sample
Three-year olds	1,530	1,029	2,559
Four-year olds	1,253	855	2,108
Total	2,783	1,884	4,667

The Third Grade Follow-Up to the HSIS was based largely on work that had already been completed for the HSIS, and provided an opportunity to examine the degree to which the impacts of Head Start on initial school readiness are changed by children’s third grade school experiences and the family/community factors associated with the child during the school years. This study provided the means to broaden the scope of analysis to include factors that had not yet been examined as part of the HSIS. The Third Grade Follow-up data collection included parent interviews, child assessments and child self-description questionnaires, teacher and principal surveys, and teacher ratings of children. The Third Grade Follow-up continued to collect HSIS comparable data on the entire sample of 4,667 children and families through the children’s third grade year.

The Tracking of Former Participants in the HSIS was designed to contact and keep in touch with families, to maintain current contact information in anticipation of ACF conducting a potential follow-up study, and to ensure adequate response rates for a potential follow-up study. Data collection included parent interviews. The project continued contacting 4,243 children and their families from the original sample in spring 2009, 2010 and 2011, excluding, with ACF agreement, 230 hard refusals (those who refused if we continued to contact them) and 194 families that never participated during HSIS or the Third Grade Follow-up.

The HSIS Participants Beyond 8^th Grade follows 4,235 children and their families from the original sample, excluding those who were excluded from prior rounds and an additional 8 families who were designated hard refusals between spring 2008 and spring 2011 during the Tracking of Former Participants in the HSIS. This effort is designed to maintain these high response rates during years in which no large-scale follow-up study will be conducted. To maintain adequate sample size, telephone interviews were conducted to update the respondent's location and contact information. This data collection was conducted in 2012, 2013 and 2014.

We are seeking OMB approval for the last two years’ data collection in spring 2015 and spring 2016 for:

The HSIS Participants Beyond 8^th Grade as described above. Data collection under this project is planned to occur in the spring of 2012, 2013, 2014, 2015 and 2016, and we are currently seeking OMB approval for spring 2015 and spring 2016. A parent interview will be used to verify and update, as necessary, families’ contact information and the child’s school information. The parent interviews will primarily be conducted over the telephone with in-person follow-up as necessary. Parent interviews will take about 20 minutes to complete. The 2015 and 2016 parent interview is provided in Appendix A.

Legal or Administrative Requirements that Necessitate the Collection

No legal requirements necessitate the collection. ACF is requesting this collection

to ensure high response rates in anticipation of a potential HSIS follow-up during or soon after the high school years of the HSIS participants.

A.2 Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

The primary purpose of this data collection is to continue following the children and families to ensure high response rates in anticipation of potential follow-up data collections. This potential follow-up will be beneficial for the examination of long-term impacts of Head Start. The HSIS information collections (HSIS, Third Grade Follow-up and the tracking studies from spring 2009 through spring 2014 described in Section A.1) have been highly successful in recruiting and maintaining participants over time, achieving nearly an 80 percent response rate in all waves. The current project will build upon these prior information collections.

Many challenges are presented by a longitudinal study with a national sample. Over time, families move and become more difficult to locate. We have been successful in gaining high cooperation from respondents for all prior rounds of data collection. High response rates were achieved for the parent interviews and the child assessments as presented in Table 2.

Table 2. Percent of Parent Interviews and Child Assessments Complete by Data Collection Period

Data Collection Activity	Response Rate
Data Collection Activity	Parent	Child
Fall 2002	86%	80%
Spring 2003	83%	84%
Fall 2003 (parent interview only)	84%	--
Spring 2004	81%	81%
Fall 2004 (parent interview only)	83%	--
Spring 2005	81%	78%
Fall 2005 (parent interview only)	83%	--
Spring 2006 (3-year-old cohort)	80%	77%
Spring 2006 (parent interview only, 4-year-old cohort)	82%	--
Fall 2006 (parent interview only)	81%	--
Spring 2007 (parent interview only, 3-year-old cohort)	81%	--
Spring 2007 (4-year-old cohort)	77%	72%
Fall 2007 (parent interview only)	81%	--
Spring 2008 (parent interview only, 4-year-old cohort)	78%	--
Spring 2008 (3-year-old cohort)	78%	74%
Spring 2009 (parent interview only)	86%	--
Spring 2010 (parent interview only)	86%	--
Spring 2011 (parent interview only) Spring 2012 (parent interview only) Spring 2013 (parent interview only) Spring 2014 (parent interview only)	85% 85% 85% 84%	-- -- - --

NOTE: N=4,667through spring 2008, N= 4243 from spring 2009 through spring 2011, N=4,235 from spring 2012 through spring 2014.

Research Questions

For this study, as for thesimilar one that preceded it, there will be no research questions. The information being collected is solely for the purpose of ensuring that future data collection efforts will be successful in reaching study participants.

Study Design

For this study, as for thesimilar one that preceded it, there will be no research questions, nor any analyses to examine any research questions. The information being collected is primarily for the purpose of ensuring that future data collection efforts will be successful in reaching study participants, and in providing some limited information about children’s status that will be informative for any future data collection and analyses. The data collection involves a 20 minute interview with the parents of HSIS participants to obtain basic and follow-up information.

Universe of Data Collection Efforts

Parent Interview (Appendix A)

The parent interview is a slightly modified version of the parent interview used for the HSIS between spring 2009 – spring 2011 (Tracking of Former Participants in the HSIS) and spring 2012 – spring 2014 (HSIS Participants Beyond 8^th Grade). In spring 2015 and spring 2016 we propose slight modifications to the parent interview, which include additional questions to get more complete contact information from the primary caregiver and for the child including cell phone numbers and email address for a potential HSIS follow-up study when the child reaches 18 years of age. Also, we have added a small set of questions on parents’ perceptions of children’s school safety and engagement as well as parent’s knowledge of children’s activities (parental monitoring). These questions assess important intermediary factors that may help with the interpretation of long-term outcomes findings in a potential HSIS follow-up study. The justification and sources (when appropriate) for these questions are included in Appendix B. The additional questions do not increase the estimated time per response as the previous version of the parent interview took less time than we had originally estimated. The average time to complete the parent interview is 20 minutes.

A.3 Improved Information Technology to Reduce Burden

The parent interviews will be conducted primarily over the telephone with in-person interviews as necessary.

A.4 Efforts to Identify Duplication

In the late 1990’s, the US General Accounting Office (GAO) released two reports concluding that (1) “…the Federal government’s significant financial investment in the Head Start program, including plans to increase the number of children served and enhance the quality of the program, warrants definitive research studies, even though they can be costly”(GAO, 1998) and (2) this information need could not be met because “…the body of research on current Head Start is insufficient to draw conclusions about the impact of the national program” (GAO, 1997).

The HSIS was the first – and continues to be the only – large-scale randomized experiment testing the effectiveness of Head Start. One purpose of the HSIS was to measure the impact of Head Start on children’s early development and school readiness. The Third Grade Follow-up provided the opportunity to look at longer term effects later in elementary school. This ongoing contact and keeping in touch with families will ensure sufficient maintenance of the original HSIS sample to allow a potential, longer-term follow-up during or soon after high school.

A.5 Involvement of Small Organizations

No small businesses or other small entities will be involved in the data collection for this study

A.6 Consequences of Less Frequent Data Collection

As recommended by the Government Accounting Office (GAO) and mandated by Congress, “definitive research studies” are legislatively required to assess the effectiveness of Head Start nationally on the school readiness of participating children. Despite increasing expenditures, including an appropriation of $8.6 billion in FY 2014, in 1997 GAO indicated that “…the body of research on current Head Start is insufficient to draw conclusions about the impact of the national program” (GAO, 1997). The HSIS was the first – and continues to be the only – large-scale randomized experiment testing the effectiveness of Head Start. The follow-ups conducted with this sample in third grade and potentially in adulthood provide the most rigorous estimate of the long-term effects of the Head Start program. The Tracking of Former Participants in the HSIS produced high response rates and provided updated contact information annually from spring 2009 through spring 2011. The current project, HSIS Participants Beyond 8^th Grade (spring 2012 through spring 2016), will help to ensure high response rates in a potential follow-up study should ACF decide to implement an evaluation following the high school years. Longer periods between data collections would risk lower response rates with respondent interest possibly waning and family movement would be more difficult to track.

A.7 Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A.8 Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Public Law 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on Wednesday, May 14, 2014, Volume 79, Number 93, page 27621, and provided a 60-day period for public comment. A copy of this notice is attached as Appendix C. During this notice and comment period, 2 comments were received. Responses to these comments can be found in Appendix D.

Consultations with Experts Outside of the Study

Information concerning the HSIS was included in the “Report of the Advisory Committee on Head Start Research and Evaluation” required as part of the Head Start Amendments of 1998. Consultation meetings were held with the Advisory Committee on Head Start Research and Evaluation on January 12, 2001, June 16-17, 2003, and September 28-29, 2005. In addition, a consultation meeting was held with the Advisory Committee on Head Start Accountability and Educational Performance Measures on June 16, 2005. The Advisory Committee on Head Start Research and Evaluation met on January 25-26, 2011 to discuss the HSIS study design, findings, and plans for future work. Committee members suggested that a potential, low-cost follow-up study was important and during or soon after high school may be the best time for conducting such a study. (Advisory Committee Members are listed in Appendix E.)

Given that this data collection involves only contacting and keeping in touch with current participants, no additional consultation was conducted.

A.9 Incentives for Respondents

Many families have participated in annual HSIS data collections since 2002. In order to minimize the burden placed on families for participating in the project, to maintain contact with families, and to maximize response rates, we will provide parents with a $20 thank you for each completed parent interview. Parents have received $20 since 2003 for participating in the HSIS, the Third Grade Follow-up, the Tracking of Former Participants in the HSIS, and the first three data collections for the HSIS Participants Beyond 8^th Grade. We are currently seeking OMB approval for two additional annual data collections in spring 2015 and spring 2016. It is important to continue a cash thank you in appreciation of the parent’s participation and to maintain consistent data collection procedures.

Recent research indicates positive effects for the use of cash incentives. In an experiment, respondents at one wave of a longitudinal study were randomly assigned to receive the same $20 incentive they had received in earlier waves or a $30 incentive or a $50 incentive. “Those offered $50 had a higher response rate than those offered $20, and this positive impact persisted for at least the next four waves of biennial data collections” (Rodgers, 2011). In another study, researchers found positive effects for those who had earlier refused, “Providing incentives to respondents, who previously refused to participate in the last survey round, significantly boosted response rates, and resulted in longer interviews and more items answered” (Zagorsky and Rhoton, 2008). Researchers looked at the use of incentives to reduce nonresponse in random digit dial telephone surveys in another study and reported that for extended interviews (as opposed to screeners), “A number of studies have found that promised incentives of $15 – $35 increase response rates” (Cantor, O’Hare and O’Connor, 2008). In a recent review of the use of incentives in longitudinal surveys, Laurie and Lynn (2008) concluded, “Overall, it seems clear that the use of respondent incentives is an important element of the strategy to minimize attrition for many longitudinal surveys. The evidence suggests consistently that attrition rates would be higher in the absence of incentives”.

A.10 Privacy of Respondents

All Westat staff members sign the Westat Pledge of Confidentiality for the study. In addition, field staff members sign a Westat Confidentiality Pledge. The two Confidentiality Pledges are included in Appendix F.

For some parent respondents, the issue of privacy of information, particularly relating to address and telephone information collected for later tracing of respondents, is a matter of great concern. Participants will be assured that the information collected will be kept private to the extent permitted by law, will be used for research purposes only by the research team, and that contact name and address information and other survey data will not be given to bill collectors, legal officials, other family members, etc.

We will implement procedural steps, similar to the steps previously used in the HSIS, the Third Grade Follow-up to the HSIS, the Tracking of Former Participants in HSIS, and the first three years’ data collection for the HSIS Participants Beyond 8^th Grade to increase respondent confidence in our privacy procedures. We will generate a set of identification labels with a unique respondent ID number and bar code. These labels will be affixed to each of the data collection instruments for a respondent. The parent interview also will have a label on the front page with the respondent’s name. The respondent’s name will be on a peel-off label and the label will be removed and destroyed prior to sending the completed interview to Westat. The use of bar codes in conjunction with the numbered identification labels enables the receipt control staff to enter cases by reading the bar code with a wand, making receipt of completed interview packages also more efficient.

The Westat Institutional Review Board (IRB) approved the project, HSIS Participants Beyond 8^th Grade on February 7, 2012. The IRB approval is reviewed and updated annually. For the next IRB review, the revised parent interview will be submitted for the 2015 IRB Continuing Review.

A.11 Sensitive Questions

There are no sensitive questions in this data collection. The purpose of the interview and how the data will be used will be explained to all participants. Participants will be reassured in person and in writing that their participation in the study is completely voluntary. A decision not to participate will not affect their standing in any government program, and if they choose to participate, they may refuse to answer any question they find intrusive. All individuals’ responses will be kept private to the extent permitted by law and none of their answers will be reported to any program, agency, or school but will be combined with the responses of others so that individuals cannot be identified. All interviews will take place in a setting where the respondent's privacy can be assured.

A.12 Estimation of Information Collection Burden

Table 3 presents data on the annual burden for respondents to the study, HSIS Participants Beyond 8^th Grade, for each data collection point. The burden estimates will be the same for the two years’ data collections in spring 2015 and spring 2016 for which we are currently requesting OMB approval. The total number of responses across these two years is 8,470 responses and 2,824 hours are the total hours requested across the two years’ data collections. The annual burden hours requested is 1,412 hours.

Table 3. Estimated Annual Response Burden for Respondents in the Study: Head Start Impact Study Participants Beyond 8^th Grade

Instrument

Total Number of Respondents

Annual Number of Respondents

Number of Responses Per

Respondent

Average Burden Hours Per Response

Annual Burden Hours

Average Hourly Wage²

Total Annual Cost

Parent Interview

8,470

4,235

1/3

1,412

$16.09

$22,719.08

Totals

4,235

1,412

$16.09

$22,719.08

Total Respondents for each Year: 4,235

Total Responses for each Year: 4,235

Total Burden Hours for each Year: 1,412

A.13 Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A.14 Estimate of Cost to the Federal Government

The total cost for the data collection activities under this current request will be $1,681,293. Annual costs to the Federal government will be $840,647 for the proposed data collection under this OMB clearance number (0970-0229).

A.15 Changes in Burden

This request is for an additional two years of data collection under OMB #0970-0229.

A.16 Plan and Time Schedule for Information Collection, Tabulation and Publication

Analysis Plans

Since this study is not designed to answer any research questions, there are no plans for statistical analysis of these data. A single indicator, the percentage of families successfully contacted, will be created to indicate the success of the overall effort to advise the study for a possible future data collection.

Time Schedule and Publication

Currently, we are requesting OMB approval for the last two year’s data collections for this project to take place in spring 2015 and 2016. Data collections are planned for mid-March through mid-May, the same months as data collection for the first three years of the HSIS Participants Beyond 8^th Grade in spring 2012, 2013 and 2014. No publications will be created from this project.

A.17 Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 Data to define these strata were obtained from Children’s Defense Fund (1999), Seeds of Success report.

2 For the HSIS respondents, about 1/3 of the respondents fall into each educational level category--less than high school degree, high school degree, and some college or associate degree. The Bureau of Labor (New Release, Usual Weekly Earnings of Wage and Salary Workers, Third Quarter 2014) reports the median weekly salary for a worker with less than a high school degree is $488, the median salary for a worker with a high school degree is $681, and the median salary for a worker with some college or associate degree is $762. The overall average hourly rate across the three categories is $16.09.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Camilla Heid
File Modified	0000-00-00
File Created	2021-01-25

OMB Supporting Statement Part A HSIS Tracking