U.S. Department of Health and Human Services
Office of Planning, Research & Evaluation
Administration for Children and Families
Tracking Head Start Impact Study Participants
Beyond 8th Grade
0970-0229
Office of Management and Budget
Clearance Package Supporting Statement B
February 8, 2012
The respondent universe and sampling methods for the three prior studies are as follows:
1. The Head Start Impact Study sampling universe was all Head Start grantees/delegate agencies operating in all 50 states, the Commonwealth of Puerto Rico, and the district of Columbia that do not specifically target the following special populations: grantees/delegate agencies serving migrant children; Head Start programs operated by Tribal organizations; children enrolled in Early Head Start; new grantees/delegate agencies (i.e., grantees/delegate agencies that are in operation for less than two years); and programs involved in FACES 2000. The starting point for creating this initial population of Head Start grantees/delegate agencies was the 1999-2000 Program Information Report (PIR) database maintained by ACYF. Migrant and Tribal Organization grantees/delegate agencies can be readily identified from this database, and “new” programs were identified as those grantees/delegate agencies that were listed in the 1999-2000 PIR but which were not listed in the 1998-1999 PIR (i.e., eliminating grantees/delegate agencies that were in operation for approximately less than two years). Early Head Start children were identified and excluded once the sample of grantees/delegate agencies had been selected. In fall 2002, a sample comprising 4,667 children and their families was randomly selected, from the applicant lists to a sample of Head Start centers. This sample was randomly assigned within centers (in a few cases, within programs) to either a treatment or a control group in a sample size ratio of 1.5. (See Section A. 2 for details of numbers of three- and four- year olds randomly assigned to the Treatment and Control groups.) If OMB is interested, we can provide the complete HSIS sampling plan.
2. The Third Grade Follow-up to the Head Start Impact Study sample continued to use only the 4,667 children and their families selected for the Head Start Impact Study.
3. The Tracking of Former Participants in the Head Start Impact Study continued tracking 4,243 children and their families who had participated in the HSIS and Third Grade Follow-up studies in spring 2009, 2010 and 2011. This sample excluded, with ACF agreement, 230 hard refusals (those who adamantly refused and/or threatened if we continued to contact them) and 194 families that never participated during HSIS or the Third Grade Follow-up.
The respondent universe and sampling methods for the current study for which we are seeking OMB approval is:
4. The Tracking Head Start Participants Beyond 8th Grade will consist of 4, 235 children and their families who participated in the HSIS, Third Grade Follow-up and/or the Tracking of Former HSIS Participants, excluding with ACF agreement, only 8 additional hard refusals identified during the spring 2009, 2010 and/or 2011 data collections. We expect to achieve response rates very close to the 85 – 86 percent achieved in spring 2009-2011.
Table 4 below provides the sample sizes for the original Head Start Impact Study, broken out by age cohort and treatment condition (Section B1 above describes the sampling procedures), with the anticipated sample sizes for the current Tracking Head Start Impact Study Participants Beyond 8th Grade. An attempt will be made to interview a parent of each of the 4, 235 children during each data collection period, with the expectation of reaching the 85 – 86 percent response rates achieved in 2009 – 2011.
Fall 2002
|
Expected sample size for continual follow-up |
|
Parent Interview |
|
|
Head Start Treatment Group |
2,783 |
2,589 |
age 3 |
1,530 |
1,441 |
age 4 |
1,253 |
1,148 |
Control Group |
1,884 |
1,646 |
age 3 |
1,029 |
903 |
age 4 |
855 |
743 |
Combined Treatment & Control |
4,667 |
4,235 |
Field staff will be used to conduct the telephone parent tracking interviews (and in person when necessary) for the Tracking Head Start Impact Study Participants Beyond 8th Grade as they have been used in each round of the Head Start Impact Study and follow-ups to date. When cases are fielded for the parent tracking interview, Westat prepares child profile reports, which include pertinent information on the families’ current telephone number, home address, work number, as well as telephone numbers of alternate contacts from the most recent locating information collected. In many cases this information is sufficient to easily reach the family. In cases where the telephone number is disconnected, Westat interviewers use the methods listed below to trace respondents. Tracing proceeds by following leads in a specific order, narrowly focused at first and then expanding as needed. Site coordinators are instructed to first use resources that are available from their homes before going into the field to physically track down the respondent. These resources, presented in the order to be implemented, include the following:
Calling alternate contacts. Telephone numbers and addresses of alternate contacts have been collected during each fall and spring data collection for both the Head Start Impact Study and the Third Grade Follow-up, and then each spring for the Tracking of Former Participants in the Head Start Impact Study through spring 2011. Parent respondents identified these contacts during the full parent interviews and the shorter tracking interviews with the understanding that these people would be contacted if we have difficulty reaching the respondent. If the alternate contact cannot provide a new telephone number or address for the respondent, interviewers ask if they know of anyone else who might know how to reach the respondent, if they know where the respondent works or is attending school, if they know the school the child attends, and whether the respondent might have married and taken a new name.
Telephone Directory. Local telephone directories can be excellent resources. Interviewers can first see if the respondent is listed at another address in or near the city. If calls to those numbers are unsuccessful and the surname is unusual, we call all of the others with the same last name and ask if they know the respondent. If the last name is common (e.g., Jones or Smith), interviewers look for a corroborating piece of information (e.g., the respondent’s middle name or a relative’s name) and call those numbers. They also try others with the same last name who live on the same street or very close to the respondent’s last address as family members often live in the same area.
Directory Assistance. Directory assistance often provides a new telephone number for families who have moved. Interviewers start with the respondent’s name (or in some cases a contact’s name) or relative’s name or branch out to only the last name. Some telephone numbers are listed under the child’s name so this is also a recommended name to try.
Sending letters to respondents in care of alternate contacts. When we have only an address for an alternate contact but no working telephone number, site coordinators may send a letter to the respondent in care of the alternate contact, informing the family that we are trying to reach them for the current round of data collection and providing a local or toll-free number the respondent can call to schedule an appointment.
Internet directories. Field staff can access search engines such as www.theultimates.com, which searches a number of directory databases simultaneously. Westat also subscribes to Lexis-Nexis, a very powerful Internet tool for locating and tracking difficult cases. It was used very effectively in the Third Grade Follow-up and the Tracking of Former Participants in the Head Start Impact Study.
Contacting neighbors via telephone. We also use Internet search engines to look up the telephone numbers of families who live on the same street as the respondent. Interviewers can call these neighbors to determine whether the family still lives at that address or to find out where the family moved. We have found that neighbors are often willing to provide this information over the telephone.
After lower cost at-home tracing methods have been exhausted, interviewers are instructed to conduct field tracing. Primary sources of information are the people who may know the respondent and who are most likely to have personal knowledge of the parent/primary caregiver’s whereabouts; they are therefore the most significant and productive tracing sources. Even if they are unable to give the interviewer an exact address, they will often provide clues that will eventually lead to a more knowledgeable source. They may also be able to provide the names, and possibly telephone numbers, of family members or friends of the family. Primary sources include the following:
1. Current residents at the respondent’s last known address. While interviewers are at the family’s last known address, they may ask the current resident if he or she knows the family’s current location. If the current resident bought the home from the respondent, the current resident may be able to provide the name of a lawyer or real estate agent who might know the respondent’s new location. If the original address is a rental property, the landlord may be able to provide the family’s current address.
2. Other neighbors. If other neighbors living within a few houses or apartments of the original address are available, interviewers may also contact them regarding the respondent’s new whereabouts.
3. Postal Service. Local post offices can be good sources of tracing information. If the person at the front counter is busy with customers, interviewers may ask to speak with someone in the back who has more contact with the carriers and may know the customers on various routes. Our experience has found that post offices in rural areas are often most helpful in locating respondents.
4. Schools. Schools may also provide information on a family’s current address or telephone number. Contacts with the schools are typically made by the site coordinator. Even when schools are not willing to release this information, they will often allow the site coordinator to write a note to the parent to be sent home with the child.
These procedures capitalize on the extensive experience, trust, and rapport achieved by field staff to form positive long-term relationships with the families and children selected for the study. Over the years this relationship has been important in obtaining parent cooperation and participation. Many study staff have worked with the same families since the beginning of the study, and have built trusted and comfortable relationships with them.
Retention rates for field supervisors are very high and generally are in the 90 to 100 percent range from one spring data collection to the next spring data collection with the retention rates for interviewers running about 85 to 90 percent. Highly experienced, skilled and trained field teams consisting of site coordinators and interviewers will be used under the supervision of the tracking study’s operations director. The operations director will implement the data collection plan and ensure monitoring and quality control tasks are conducted in an efficient, organized, and timely manner to continue the collection of high quality tracking data and achieve the high response rates consistently maintained through the various components of the HSIS. Site coordinators will serve in a pivotal role as the primary local contact, enlisting cooperation and maintaining participation of respondents; coordinating all tracking activities in the site; managing field interviewers; and ensuring quality control. They will report to the central office operations director and her staff. The number of field interviewers and the time allocated will be adjusted according to the number of cases and their location within the site.
Locating families for tracking is the primary goal of this data collection. We will employ several strategies to minimize sample attrition and obtain a high response rate. Minimizing attrition requires continuing to engage families who have participated in the past and convincing families who have not participated in recent rounds to rejoin the study. Both of these components, required to minimize attrition, are discussed below.
Engaging Current Participants. To keep respondents interested in continuing their participation, we will continue to emphasize that the Head Start Impact Study was and continues to be a ground-breaking and important study. Advance letters will be mailed out that explain the study’s significance, the importance of continued participation, the fact that participation is voluntary, the incentives for participation that will be provided, the assurance of privacy, and the fact that participation will not affect their current benefits. A copy of the advance letter is included in Appendix E. Incentives can motivate parents to participate by demonstrating the extent to which we value their time and insights. We will provide an incentive of $20 for each respondent to acknowledge the value of their continued participation as a respondent for this study.
Converting Families Who Have Refused in the Past. Through every data collection of the HSIS, Third Grade Follow-up, and Tracking, the interviewers tried to contact each family who was randomly assigned to the study. The only exceptions were cases that had been determined to be “hard refusals.” These were individuals who told us—in no uncertain terms—that they did not wish to participate in the study and did not want to hear from a representative of the HSIS again. While respondents can be encouraged and often persuaded to participate, participation is voluntary. It is important for the integrity of the study that respondents participate at will. For the earlier Tracking of Former Participants in the Head Start Impact Study, we followed a sample of 4,243 in spring 2009 through spring 2011, excluding the 238 hard refusals and those 194 families that had never been located or had never completed an interview throughout the course of the study. We will try to re-engage 4,235 families for this tracking effort, excluding only the 8 additional hard refusals from spring 2009 through spring 2011.
Field interviewers and site coordinators are trained in strategies for gaining and maintaining cooperation. These include: (1) being prepared and confident in responding to questions regarding the study; (2) being enthusiastic about the study; (3) listening attentively to what the respondent is saying; (4) maintaining a pleasant, friendly, professional attitude and emphasizing the positive as opposed to arguing; (5) offering options and displaying flexibility in accommodating parents’ schedules; (6) letting the respondent know how important he or she is and how important the study is; and (7) emphasizing the confidentiality of responses.
Field staff members also are trained in identifying, understanding, and responding appropriately and effectively to reasons why parents may be hesitant to cooperate. They are taught strategies to be empathetic with the parents, reassure them and ensure their cooperation including: (1) establishing the purpose of the call by identifying themselves as calling in connection with their child’s participation in a national study and referring to the child and the name of the child’s school to avoid respondents concluding that this is a telemarketing call; (2) verifying the legitimacy of the study and giving respondents key information such as the interviewer’s name, site coordinator’s name, Westat’s name, the study name, and the toll-free number; (3) establishing rapport and asking questions during the introduction that elicit a “yes” response, making it harder for the respondent who agrees with you to refuse later, (4) offering token agreement and understanding of the respondent’s viewpoint, e.g., “I can understand that” or “you certainly have the right to feel that way” to establish rapport and a sympathetic ear and disarm negativity; and (5) reassuring the respondent that the entire interview is private and individual responses will not be revealed. Field staff members are trained to try to find out why the respondent is reluctant and to address these concerns. Just as one respondent differs from another, the reasons for refusals are many and varied. Therefore, it is necessary to train field staff to become sensitive to how firm a “no” they are receiving and sense the reasons behind the hesitancy to develop ways to reassure the respondent and gain cooperation. If field staff still find that they are not able to convince a parent to participate in the interview, they are trained to try and end on a friendly note and lay the groundwork and leave the door open for someone else to make another attempt in future, an approach that often has proven very effective.
The tracking instrument used in this study has been used in prior rounds of tracking for the Head Start Impact Study, the Third Grade Follow-up and the Tracking of Former Participants in the Head Start Impact Study and has been effective at maintaining high response rates.
The contractors who worked on the initial design for the Head Start Impact Study (28200-0022) and the Third Grade Follow-up (contracts 23302-008706TK001 and HHSP23320062929YC), include Westat (prime contractor), Chesapeake Research Associates, Abt Associates, Urban Institute, and the American Institutes for Research. For the Tracking of Former Participants in the Head Start Impact Study, no statistical analysis of the data was required and it is not anticipated to be needed for the Tracking Head Start Impact Study Participants Beyond 8th Grade, but should it be required, Westat statistical staff will be employed as necessary for this contract.
Frank Jenkins, Senior Statistician, Westat, (301) 279-4502.
Cantor, D., O’Hare, B. and O’Connor, K. (2007) “The Use of Monetary Incentives to Reduce Non-Response in Random Digit Dial Telephone Surveys” pp. 471-498 in J. M. Lepkowski, C. Tucker, J. M. Brick De Leeuw, E., Japec, L., Lavrakas, P. J., Link, M. W., & Sangster, R. L. (Eds.), Advances In Telephone Survey Methodology, New York: J.W. Wiley and Sons, Inc.
Laurie, H. and P. Lynn (2009) “The Use of Respondent Incentives on Longitudinal Surveys.” pp. 205 – 231, in P. Lynn (ed) Methodology of Longitudinal Surveys” John Wiley & Sons: Southern Gate.
Rodgers, Willard L. (2011) “Effects of Increasing the Incentive Size in a Longitudinal Study.” Journal of Official Statistics, Vol. 27, No. 2, 2011, pp. 279-299.
Schulman, K., Blank, H., and Ewen, D. (1999). Seeds of success: State prekindergarten initiatives 1998-1999. Washington, DC: Children’s Defense Fund.
U.S. General Accounting Office. (1997). Head Start: Research provides little information on impact of current program. Washington, DC: U.S. Government Printing Office.
U.S. General Accounting Office. (1998). Head Start: Challenges in monitoring program quality and demonstrating results. Washington, DC: U.S. Government Printing Office.
Zagorsky, Jay and Rhoton, Patricia (2008), “The Effects of Promised Monetary Incentives on Attrition in a Long-Term Panel Survey” (Fall 2008). Public Opinion Quarterly, Vol. 72, Issue 3, pp. 502-513.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Department of Health and Human Services |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |