Head Start CARES
Supporting Statement for OMB Clearance Request
May 27, 2008
Table of Contents
part a 1
justification 1
A1. Circumstances Making the Collection of Information Necessary 1
Legal or Administrative Requirements that Necessitate the Collection 2
Study Objectives 2
A2. Purpose and Use of the Information Collection 3
A3. Use of Improved Information Technology and Burden Reduction 4
A4. Efforts to Identify Duplication and Use of Similar Information 4
A5. Impact on Small Businesses or Other Small Entities 4
A6. Consequences of Collecting Information Less Frequently 4
A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 4
A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 4
A9. Explanation of Any Payment or Gift to Respondents 5
A10. Assurance of Confidentiality Provided to Respondents 5
A11. Justification for Sensitive Questions 5
A12. Estimates of Annualized Burden Hours and Costs 5
Estimates of Annualized Costs 6
A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 7
A14. Annualized Cost to the Federal Government 7
A15. Explanation for Program Changes or Adjustments 7
A16. Plans for Tabulation and Publication and Project Time Schedule 7
A16.1 Analysis Plan 7
A16.2 Time Schedule and Publications 7
A17. Reason(s) Display of OMB Expiration Date is Inappropriate 7
A18. Exceptions to Certification for Paperwork Reduction Act Submissions 7
Part B: Collection of Information Employing Statistical Methods 1
B1. Respondent Universe and Sampling Methods 1
B2. Information Collection Procedures 1
B3. Methods to Maximize Response Rates 2
B4. Test of Procedures 3
B5. Individuals Consulted on Statistical Aspects of Design 3
AppendiCes 1
A: Federal Register Notice
B: Project Description
C: Initial Call Script
D: Recruitment Phone Screener
E: Site Visit Discussion Guide
The Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS) funds the Head Start Program. Head Start is designed to “narrow the gap” between disadvantaged children and their more affluent peers through comprehensive programming during the pre-school period designed to improve children’s social competence and academic readiness for school. ACF has contracted with MDRC to conduct the Head Start CARES (Classroom-based Approaches and Resources for Emotion and Social Skill Promotion) research project.
This section provides supporting statements for each of the eighteen points outlined in Part A of the OMB guidelines and the five points outlined in Part B of the guidelines for the collection of information for Head Start CARES. This submission seeks clearance for the site recruitment materials.
A subsequent OMB package will seek approval for instruments to collect baseline data and follow-up program outcomes data from teachers, classrooms, children and families and for field research instruments for the implementation study.
ACF seeks approval for the data collection activities described in this request in order to support the Head Start CARES study. Head Start CARES will develop and implement an evaluation of specific program enhancements within Head Start centers that are intended to improve children’s social-emotional development.
Head Start is the nation’s largest federally sponsored early childhood education program, and today it serves nearly 1 million low-income children. The program was designed to enrich the formative experiences of these children so that they might better be able to overcome the challenges associated with economic disadvantage. From its inception, the primary focus of the program has been on enhancing children’s physical, cognitive, and social-emotional development.
A wealth of developmental research highlights the deleterious consequences of poverty for young children. More specifically, because they are exposed to a wide range of psychosocial stressors, low-income children have been found to be at greater risk for developing emotional and behavioral difficulties than their middle-income peers.1 Empirical studies have documented prevalence rates of emotional and behavior problems among preschool children as high as 20%2 and even higher rates among children in poverty, with as many as 40% of children demonstrating behavioral difficulty.3 Accordingly, Head Start programs and other preschools in low-income communities report a pressing need for effective tools for building children’s social-emotional skills. Indeed, the payoff of such prevention efforts may be high, as supporting low-income children’s healthy emotional and behavioral development during the preschool years is likely to influence their chances for success in school and beyond, both for the highest risk children in the classroom as well as for their lower risk peers.
Within the guidelines of the National Head Start Performance Standards, Head Start programs have enjoyed a great deal of flexibility to adapt curricula and other program characteristics as needed to best serve Head Start children and families. Although Head Start has long been a leader in innovative early childhood practices and initiatives, these practices rarely have been evaluated in large-scale, rigorously implemented studies. More research is needed to establish the effectiveness of specific programs or practices targeting specific skill development.
There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.
The study will be a large-scale evaluation that will test different strategies to enhancing social-emotional development. These strategies build on distinct theories of change about how to intervene effectively to support children’s social and emotional development, but the ultimate goal of each is the same: they all are targeted at promoting children’s social-emotional competence in the classroom setting. In each case, teachers are trained in the new approaches and then are provided with mentors to implement the new strategies in the classroom context.
To date, these strategies have been tested in limited circumstances, such as in single cities or in small numbers of programs that have chosen to adopt the intervention. Never have multiple social-emotional programs been tested on a large-scale in the context of a federal initiative.
The project will use a group-based randomized design to test the effects of a set of evidence-based strategies designed to improve the social and emotional development of children in Head Start classrooms. The evaluation will randomly assign approximately 120 Head Start centers that choose to participate in the study to either one of the social-emotional interventions or a control group that will offer “business as usual” programming.
The project as designed holds the promise of identifying the impacts of these new approaches compared to current practices within Head Start settings and providing lessons about how they can best be integrated into Head Start classrooms around the country.
The following research questions will be addressed by the study:
What are the impacts of different interventions targeted on children’s social and emotional development, compared to current practice within the Head Start settings? What are the pathways by which those interventions produce their impacts?
How do the effects of each intervention vary by characteristics of Head Start grantees,4 classrooms, teachers, families or children?
What characteristics of Head Start settings are necessary for effective implementation of the intervention tested?
What features of Head Start settings are associated with successful training, technical assistance, and implementation of the interventions to be studied?
By addressing these questions, the Head Start CARES project will provide the information that federal policymakers and Head Start administrators can use to increase the program’s capacity to improve the social-emotional skills and school readiness of pre-school aged children.
Data collection for this study will be carried out in two phases:
Phase 1—recruitment of Head Start grantees and centers into the study; and
Phase 2—data collection in selected study sites.
This submission seeks clearance for the Phase 1 site recruitment materials: a project description, an initial call script, a recruitment phone screener, and site visit discussion guides. These tools will be used to screen and recruit an eligible sample of 15-25 Head Start grantees and 120 associated Head Start centers. The site recruitment process will ensure that the centers included in the study meet the study criteria, are well operated and provide a stable environment for both program implementation and data collection. The recruitment materials will enable the contractor to identify good candidates, as well as sites that should be excluded from the study.
Site recruitment will involve the following steps: after the contractor draws a sample of Head Start grantees (see Part B), staff will send the grantee administrator a project description and schedule a call. After describing the project, the screener will be administered to gather additional information not available through other sources in order to determine if the grantee is eligible for the study. If the initial screening indicates the grantee is a viable site and is interested in participating, a site visit will follow. During the site visit, contractor staff will meet with staff at the grantee and the Head Start centers to confirm the accuracy of information gathered to date and to learn more about the characteristics of the centers, the staffing arrangements at the centers, the education programs in operation, and the characteristics of each classroom that would be included in the study.
The 120 centers ultimately selected will be randomized to treatment groups that will implement the interventions being tested or a control group that will not implement these interventions.
The information will be collected through semi-structured discussions that are not conducive to information technology, such as computerized interviewing.
The information collection requirements for the site recruitment activities will build on sources already available. To reduce duplication and burden, the project will first attempt to secure information through the Head Start Program Information Reports (PIR) and the Office of Head Start Monitoring System. These contain information (i.e., number of centers) that will help identify programs that should be included in the study sample, as well as indicate if a program is a poor candidate for the study (i.e., are in deficiency status, first year of operation.) The contractor will pre-populate fields in the phone screener instrument that can be obtained from the administrative data. Then the contractor will confirm the administrative data elements obtained are up-to-date and accurate.
Some information needed to ascertain if a program is appropriate for inclusion can only be collected through conversations with the programs. The documents for which ACF seeks clearance complement the information that will be gathered through current data sources.
Not applicable. No small businesses are expected to be involved.
The project team will collect site recruitment information only once. This information collection will occur in two stages. The phone screen will determine if the program is suitable for the study. The site visits will be conducted with those sites that are deemed viable.
There are no special circumstances for the proposed data collection.
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and the Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on March 31, 2008, Volume 73, Number 62, page 16861, and provided a sixty-day period for public comment. A copy of this notice is attached as Appendix A.
During the notice and comment period, the government received nine requests for copies of the instrument. All requests were fulfilled. All comments received were positive, expressing interest in participation.
The staff of MDRC, the evaluation contractor hired by ACF, and their sub-contractor, The Lewin Group, who have substantial knowledge of Head Start programs, contributed to the discussion guide.
No payments to respondents are proposed for this information collection.
The site recruitment data collection instruments will not collect information that is considered proprietary, thus precautions to protect confidentiality are not necessary.
There are no personally sensitive questions in this data collection.
This proposed information collection does not impose a financial burden on respondents. Respondents will not incur any expenses other than the time they spend reviewing the project description, and answering questions contained in the phone screen and the site visit discussion guide.
Exhibit A12.1 summarizes reporting burden on respondents to the Initial Script, Project Description, the Phone Discussion Points and Screener, and the Discussion Guide for Site Visits. The annual burden is estimated from the total number of completed discussions and the minutes taken to complete the discussions. Thus, the total burden is expected to be 510 hours.
Exhibit A12.1: Site Recruitment Information Collection Activities
Instrument |
Number of Respondents |
Number of Responses per Respondent |
Average Burden Hours per Response |
Total Burden Hours |
Initial Call to Grantee |
60 |
1 |
0.25 |
15 |
Project Description |
60 |
1 |
0.25 |
15 |
Phone Discussion Points & Screener |
60 |
1 |
1 |
60 |
Discussion Guide for Site Visits: Grantee |
30 |
1 |
2 |
60 |
Discussion Guide for Site Visits: Centers |
180 |
1 |
2 |
360 |
Estimated Total Annual Burden Hours |
|
510 |
||
Respondents will be Head Start grantee directors and Head Start center coordinators. The computed total estimated annual cost takes into account two different average hourly wages—one for program administrators and one for Head Start center coordinators.
To compute the total estimated annual cost, the total burden hours were multiplied by the average hourly wage for two labor categories: Head Start directors (grantee level) and education coordinators (center level).
The source used to determine the wage was the Head Start Program Information Reports. The total estimated annual cost is $11,995. The estimate assumes the contractor will call and describe the project and send the project description to up to 60 Head Start directors. The contractor will screen the directors via phone. Of the 60 grantees, up to 30 will be visited. While on site, the contractor will meet with the 30 directors. Within these 30 sites, the contractor will also meet with up to 6 Head Start education coordinators. Thus, the estimate assumes the contractor will meet with at most staff in 180 centers across the 30 sites.
Instrument |
Total Burden Hours |
Average Hourly Wage: Head Start Directors |
Average Hourly Wage: Head Start Center Education Coordinators |
Annual Cost |
Initial Call to Grantee |
15 |
$30.02 |
N/A |
$450 |
Project Description |
15 |
$30.02 |
N/A |
$450 |
Phone Discussion Points & Screener |
60 |
$30.02 |
N/A |
$1,801 |
Discussion Guide for Site Visits: Head Start Director |
60 |
$30.02 |
N/A |
$1,801 |
Discussion Guide for Site Visits: Center Education Coordinator |
360 |
N/A |
$20.81 |
$7,492 |
Total Annual Cost |
|
$11,995 |
||
Not applicable. The recruitment information collection activities do not place any capital cost or cost of maintaining capital requirements on respondents.
The site recruitment activity and associated forms have been developed by the evaluation contractor, MDRC, in performance of HHS Contract Number HHSP23320072909YC . The annualized cost to the government for this task is $424,137.
This submission to OMB is a new request for approval.
Site recruitment involves two-stages of information collection. The first is phone calls to grantee administrators to gather basic information about the program to determine if it is a viable study site. Following each call, the contractor will summarize the discussion in writing and make an initial assessment regarding the potential for working further with the grantee. Information will be added to a tracking sheet.
Senior contractor project staff will make the recommendations to ACF on whether to conduct a site visit to a grantee or whether to exclude it from the study. Following each site visit, the site visit team will summarize the key points, highlighting any issues that might compromise the site’s ability to participate. Once the visits are completed, senior staff will review the site visit notes and make final recommendations to ACF about study sites.
Site recruitment will occur over two years. No specific publications will emerge from this effort.
All instruments for the Head Start CARES project will display the expiration date for OMB approval.
No exceptions are necessary for this information collection.
The Head Start CARES project will randomly assign approximately 120 Head Start centers to either a treatment group that will receive one of the social-emotional interventions or a control group that will offer “business as usual” programming. The universe of centers is 18,875. The starting point for creating the initial population of Head Start grantees will be the Program Information Report (PIR) database maintained by ACF.
Sampling Plan.
The study’s sampling plan will provide a sample of Head Start grantees and centers that represent a large and diverse portion of the Head Start student population and provides adequate statistical precision for estimating intervention effects. The plan is designed to produce a sample of approximately 15-25 grantees within which Head Start centers will be randomized to intervention groups or a control group.
The proposed sampling plan will share some important features with the sampling plans used for the Head Start Impact Study and the Head Start Family and Child Experiences Survey (FACES). The first step in our sampling process will be to define a population of Head Start grantees and centers for inclusion in the study. All grantees will have an adequate number of centers and students to support the experimental design. The design includes three interventions to be tested (treatments) and a control group. Therefore, all grantees in the sample must have at least four centers that are eligible, able and willing to participate in the study. The PIR will be used to provide a sampling frame (or list) of grantees once final decisions are made about criteria for inclusion in the study population.
The next step in the sampling process will be to stratify all grantees in the sampling frame. These administrative entities will be stratified by a combination of: (1) the demographic composition of their student enrollment, (2) the ACF region in which they are located, and (3) the urban/suburban/rural nature of their location. (These criteria were used to stratify the sample for the Head Start Impact Study.) The number of strata constructed will equal the number of grantees to be sampled and one grantee from each stratum will participate in the study.
Within each stratum the contractor will randomly select a “first-choice,” “second choice,” and “third choice” grantee, with probability proportional to their total student enrollment (size). Grantees then will be recruited from each stratum in the order they were chosen until one grantee is brought into the study from each stratum.
The site recruitment study task relies on the data collected through the instruments for which clearance is being requested. The recruitment process is designed to both stimulate Head Start grantee and center interest in participating in the study and assess their potential. The recruitment process will involve discussions with the following stakeholders:
Head Start grantee directors
Head Start center coordinators
The initial contact will be with the grantee. The contractor plans to call up to 60 grantees. Prior to scheduling a call, the contractor will send the grantee director a project description. After describing the project, a screener will be administered to confirm administrative data and gather additional information about the Head Start program. This includes the number of centers operated by the grantee, funded and current enrollment, programs or curricula currently offered that focus on children’s social-emotional development, changes anticipated in the near term, and whether the grantee is participating in another evaluation. The screener also explores characteristics of each center, including number of classes, number of children per class, classroom hours of operation, whether the center operates year-round, and if the center has a home-based option.
If the site remains a viable option, a site visit will follow. The first meetings will occur with the Head Start grantee director. The discussion topics include an overview of key aspects of the study, the characteristics of the grantee and centers, staffing of centers, components of the education program, and next steps for the study. The research team will then meet with center-level coordinators. The topics are similar, and focus specifically on the center. There is also a classroom observation component.
It is expected that some grantees will not be eligible for the study or will choose not to participate. The contractor plans to conduct site visits to 30 grantees in order to obtain the final sample between 15-25 grantees. While on site, the contractor will meet with the 30 grantee directors. Within these sites, the contractor will also meet with the Head Start center coordinators. The estimate assumes the contractor will meet with staff in up to 180 centers across 30 sites.
The project description is included in Appendix B. The phone script is Appendix C. The screening instrument is included in Appendix D. The site visit discussion guide is included in Appendix E.
Site recruitment will be an intensive effort. The recruitment approach is based on establishing strong partnerships with the grantees and actively addressing potential concerns. This will involve:
Early and ongoing communication and support provided from the Office of Head Start and Regional Office staff.
Gaining the support and endorsement of the National Head Start Association.
Contacting sampled grantees in multiple phases (a phone screen and site visit). Grantees will have the opportunity to think about the study and formulate questions between the initial call and the site visit. The on-site meetings will not only further validate that sites meet the criteria for inclusion in the study, but will provide an opportunity to explain the study and motivate grantee and center staff to participate. The visits also provide an opportunity for mutual learning and problem solving.
The contractor is working closely with the Office of Head Start. Senior contractor staff met with OHS staff to describe the study and discuss recruitment strategies. OHS staff stressed the importance of involving the Regional Program Managers. Contractor staff will participate in one or more of the monthly Regional Program Manager calls to describe the study and ensure that Regional Program Managers have the necessary information should grantees contact them directly regarding the study. The contractor has been in contact with the National Head Start Association to describe the study.
Once recruitment efforts begin, the contractor will be persistent in attempts to reach the sampled grantees via phone. Contractor staff will keep a log of all phone calls and emails to grantees and will keep ACF apprised of any issues that emerge. The contractor will also keep the Regional Program Managers informed of the progress in recruitment.
We have not field tested the phone screener or the discussion guide. The structure of the instruments is similar to those used in other non-federal, foundation funded studies conducted by the contractor with similar respondents.
Both instruments serve as a guide for conversations between the contractor and grantee staff. Contractor staff will be able to ask clarifying questions to assure grantees are responding appropriately and that we are correctly interpreting their responses. The response time estimates are based on past experience with similar effort.
Howard Bloom from MDRC is leading the research design subtask for MDRC. On April 7, 2008 MDRC held a conference call with a sub-group of the study Technical Working Group with expertise on group randomized trials (GRT). The GRT subgroup includes: Larry Hedges (Northwestern University), Jeff Smith (University of Michigan), Stephen Raudenbush (University of Chicago), Mark Lipsey (Vanderbilt University) and Tom Cook (Northwestern University).
1 Dodge, K., Petite, G., & Bates, J. (1994). Socialization mediators of the relations between socioeconomic status and child conduct problems. Child Development, 65, 649-660. Fantuzzo, J., Stoltzfus, J., Lutz, M.N., Hamlet, H., Balraj, V., Turner, C. & Mosca, S. (1999). An evaluation of the special needs referral process for low-income preschool children with emotional and behavioral problems. Early Childhood Research Quarterly, 14(4), 465-482., Farmer, E.M.Z, Stangl, D.K., Burns, B.J., Costello, E.J., & Angold, A. (1999). Use, persistence, and intensity: Patterns of care for children’s mental health across one year. Community Mental Health Journal, 35, 51-46.
2 Campbell, S.B. (1995). Behavior problems in preschool children: A review of recent research. Journal of Child Psychology and Psychiatry, 36, 113-149. Lavigne, J.V., Gibbones, R.D., Christoffel, K., K.Arend, R., Rosenbaum, D., Binns, H., Dawson, N., Sobel, H. & Isaacs, C. (1996). Prevalence rates and correlates of psychiatric disorders among preschool children. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 204-214.
3 Qi, C.H., & Kaiser, A.P. (2003). Behavior problems of preschool children from low-income families. Topics in Early Childhood Special Education, 23, 188-216.
4 A grantee is the entity that receives a grant to operate a Head Start program; a grantee administers one or more centers with one or more classrooms.
| File Type | application/msword |
| File Title | head start cares |
| Author | KAREN.GARDINER |
| Last Modified By | lsupplee |
| File Modified | 2008-06-02 |
| File Created | 2008-05-27 |