DHHS/ACF/OPRE Head Start
Classroom-based Approaches and
Resources for Emotion and Social skill
promotion (CARES) project:
Tracking Participants
OMB Information Collection Request
0970–0364
Supporting Statement
Part A
May 2013
Submitted By:
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
7th Floor, West Aerospace Building
370 L’Enfant Promenade, SW
Washington, D.C. 20447
Project Officers:
Ann Rivera
Lauren Supplee
Table of Contents
A.1 Necessity for the Data Collection
A.2 Purpose of Survey and Data Collection Procedures
A.3 Improved Information Technology to Reduce Burden
A.4 Efforts to Identify Duplication
A.5 Involvement of Small Organizations
A.6 Consequences of Less Frequent Data Collection
A.7 Special Circumstances
A.8 Federal Register Notice and Consultation
A.9 Incentives for Respondents
A.10 Privacy of Respondents
A.11 Sensitive Questions
A.12 Estimation of Information Collection Burden
A.13 Cost Burden to Respondents or Record Keepers
A.14 Estimate of Cost to the Federal Government
A.15 Changes in Burden
A.16 Plan and Time Schedule for Information Collection, Tabulation and Publication
A.17 Reasons Not to Display OMB Expiration Date
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
B.1 Respondent Universe and Sampling Methods
B.2 Procedures for Collection of Information
B.3 Methods to Maximize Response Rates and Deal with Nonresponse
B.4 Tests of Procedures or Methods to Be Undertaken
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Appendix A: Parent Survey
Appendix B: Head Start CARES Notices
B.1: Published 60-Day Notice
B.2: Draft 30-Day Notice
Appendix C: Public Use File – Data Masking
Appendix D: Parent Newsletter
Appendix E: Bibliography
A1. Necessity for the Data Collection
The Office of Planning, Research and Evaluation (OPRE) of the Administration for Children and Families (ACF), U.S. Department of Health and Human Services (HHS), is submitting the Request for OMB Review in support of an ongoing project to collect contact and follow-up information from children and families who participated in the Head Start Classroom-based Approaches and Resources for Emotion and Social skill promotion (CARES) project. The Head Start CARES project is a group-randomized trial of three social-emotional program enhancements within a diverse set of nationally representative Head Start classrooms. In anticipation of conducting a large-scale Head Start CARES follow-up during the fourth grade and to ensure high response rates for such a follow-up study, ACF has awarded a contract to MDRC (a policy research firm) and Survey Research Management (a survey firm; SRM) to continue contacting and tracking families for three years for the Head Start CARES Project: Tracking Participants. Currently, we are seeking OMB approval for three years of data collection from summer 2013 through spring 2016. Data collection will include telephone tracking interviews with parents for the primary purpose of maintaining updated contact information necessary for the planning of a fourth grade follow-up study. Additionally, a small set of items on parents’ perceptions of children’s social and emotional skills and behavioral outcomes during the interview will enable a brief examination of the longer-term impacts of the CARES enhancements in the early elementary school years.
OMB previously approved the following information collection activities for this project:
DHHS/ACF/OPRE Head Start Classroom-based Approaches and Resources for Emotion and Social skill promotion (CARES) project: Site Recruitment and Impact and Implementation – OMB approved the initial package for Head Start CARES site recruitment in October 2008 (OMB #0970-0357). The initial rounds of impact and implementation data collection for children and families began in the spring of 2009 and were completed in children’s kindergarten year in spring 2013 (OMB # 0970-0364, Expiration Date after a one-year extension: 05/31/2013).
We now request OMB approval of the following information collection activities:
DHHS/ACF/OPRE Head Start Classroom-based Approaches and Resources for Emotion and Social skill promotion (CARES) project: Tracking Participants – The previous HS CARES studies, listed above, have been highly successful in recruiting and maintaining participants over time. Data collection in the kindergarten year achieved an 83 percent response rate of parent reports. The Head Start CARES project: Tracking Participants is designed primarily to track participating children and families to maintain these high response rates during years in which no large-scale follow-up study will be conducted. We will conduct telephone tracking interviews (with in-person follow-up as necessary) to update the respondent's location and contact information. This information will be collected from parents or guardians. In addition, a small set of items during the interview will examine parents’ perspective on children’s social and emotional skills and behavioral outcomes. This will provide beneficial information on the longer-term impacts of the three social-emotional enhancements within Head Start classrooms in the early elementary school years before the large-scale follow-up is conducted.
Recently, researchers and policymakers have drawn attention to the high rate of emotional and behavioral difficulties among young, low-income children. Exposed to a wide range of psychosocial stressors, children in poor neighborhoods are at greater risk for developing emotional and behavioral difficulties than their middle income peers.1 This work signals the need to build and disseminate evidence about preschool classroom processes that support, rather than compromise, young children’s emotional and behavioral development, in conjunction with and in support of practices that promote their early learning.
Overview of Previously Conducted Head Start CARES Project. A number of promising program enhancements targeting children’s social and emotional skills have been implemented and studied in a range of preschool settings. At the same time, these studies have largely been conducted in ideal conditions: in single cities, with programs highly motivated to take up the intervention, and with training and technical assistance provided under the direction of senior academic researchers. The Head Start CARES project, a well-designed project with a nationally representative sample of Head Start programs and a rigorous multi-celled cluster-analytic design, provides the opportunity to identify the most effective of these new approaches for enhancing social and emotional development in the preschool years.
The study utilizes a group-based randomized experimental design to test the effects of three very different evidence-based program enhancements designed to improve the social and emotional development of three- and four-year old children in Head Start classrooms. The study aims to provide the information federal policymakers and Head Start providers will need if they are to increase Head Start’s capacity to improve the social-emotional skills and school readiness of preschool-age children.
Following is a brief description of the past studies of the Head Start CARES project:
Head Start CARES project: Site Recruitment and Impact and Implementation: The initial study design focused on site recruitment materials: a project description, an initial call script, a recruitment phone screener, and site visit discussion guides. These tools were used to screen and recruit an eligible sample of Head Start grantees and associated Head Start centers. Subsequently, instruments were approved to collect data on two cohorts of children (4 year olds only and 3- and 4-year-olds) to assess the impact and implementation of the three social-emotional program enhancements through surveys with teachers and parents, direct child assessments, as well as interviews with teachers, coaches, trainers, center directors, center staff, and grantee/delegate agency directors. The Head Start CARES study enrolled 3,961 children (3- and 4-year-olds) and their families, although 328 were excluded when left the Head Start program during the implementation year, resulting in a final sample of 3,633 (for more information on sample size see Supporting Statement B, section B1). Further, 2,673 of the 4-year-olds participants were included in the kindergarten follow-up. The 3-year-olds were not fielded in the kindergarten follow-up.
Following is a description of the current study for which we are seeking OMB approval:
Head Start CARES project: Tracking Participants investigation will follow 3,633 children and their families from the original sample, excluding the 328 who left Head Start during the intervention year. This effort is designed to maintain high response rates during years in which no large-scale follow-up study will be conducted. To maintain adequate sample size, telephone interviews will be conducted to update the respondent's location and contact information. Data collection under this contract is planned to occur in the summer of 2013, spring of 2014, spring of 2015, and spring of 2016. We are currently seeking OMB approval for summer 2013 through spring 2016. Parent tracking and follow-up surveys will be used to verify and update, as necessary, families’ contact information and the child’s school information. The parent tracking and follow-up surveys will primarily be conducted over the telephone, with in-person interviews as necessary. These surveys will take about 30 minutes to complete. The Parent Tracking and Follow-up Survey also includes brief parent-reported measures of children’s social and emotional skills and behavioral outcomes. The survey is provided in Appendix A.
There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Approach
The primary purpose of this data collection is to continue tracking the children and families to ensure high response rates because we are planning large-scale follow-up data collections in the fourth grade. This tracking and brief follow-up study will be beneficial for the examination of long-term impacts of the three enhancements implemented in the Head Start CARES project for children during or soon after the third grade. The Head Start CARES project (described in Section A.1) has been highly successful in recruiting and maintaining participants over time, tracking nearly 92% of four-year-old children into kindergarten.
In support of a further examination of outcomes in third grade and beyond, information must be collected from parents or guardians until the third grade year. To enable the opportunity to conduct data collection in third grade, complete tracking information on the full sample, for all years until third grade is necessary.
In addition to location and contact information, a small set of items will provide information on the parents’ perception of the children’s social and emotional skills and behavioral outcomes. Prior studies of promising social and emotional preschool programs have indicated a need for long-term follow-up well into the elementary school grades to understand the full scope of impacts on children (Barnett & Masse, 2007; Ludwig, 2011; Nores, Barnett, Belfiel & Schweinhart, 2005; Reynolds, Temple, Robertson, & Mann, 2002). There are a number of examples of promising preschool programs that show differing effects in the short- and long-term. Therefore, these additional survey items will help fill in the existing knowledge gap on the longer-term impacts of the three social-emotional enhancements within Head Start classrooms in the early elementary years.
Research Questions
Purposes of the data collection discussed in this extension request include the following:
To locate families and ensure that future data collection efforts will be successful in reaching study participants. This does not involve substantive research questions or analyses;
To what extent do the three social and emotional enhancements relate to longer-term impacts on children’s social and emotional skills and behavioral outcomes between kindergarten and 3rd grade?
Universe of Data Collection Efforts
Of the 3,961 children in the total sample, 246 of the 4-year-olds and 82 of the 3-year-olds moved away or changed school during preschool (the Head Start CARES implementation year). These 328 children will not be included in the current tracking and brief follow-up study. For the current tracking and brief follow-up study, the remaining 3,633 sample families will continue to be tracked in the project until children reach the third grade, because we are planning a large-scale fourth grade follow-up study. This sample is further divided into Cohort 1 four-year-olds (n=603), Cohort 2 four-year olds (n=2,070), and Cohort 2 three-year-olds (n=960). Please see Exhibit A.1 below and the accompanying detailed explanation for information about the sample. This number is based on the total number of four-year-old children fielded for kindergarten data collection; in addition, because three-year-olds did not participate in the kindergarten follow-up, we include those three-year-olds originally selected into the sample in preschool that did not leave their preschool during the Head Start CARES implementation year. In the summer of 2013 and spring of 2014, all three groups will be to be tracked. In the spring of 2015, only Cohort 2 families will be tracked. In the spring of 2016, only Cohort 2 three-year-old families will be tracked. All respondents will respond to the same survey (see Appendix A).
Exhibit A.1 Number to be Tracked each year
|
Summer 2013 |
Spring 2014 |
Spring 2015 |
Spring 2016 |
Cohort 1 (4-year-olds) |
603 |
603 |
-- |
-- |
Cohort 2 (4-year-olds) |
2,070 |
2,070 |
2,070 |
-- |
Cohort 2 (3-year-olds) |
960 |
960 |
960 |
960 |
Total Number of Respondents |
3,633 |
3,633 |
3,030 |
960 |
This document proposes to collect information necessary to identify Head Start CARES study respondents’ current location and briefly follow-up with respondents until the children reach third grade. Appendix D presents an outreach letter we will use to reach out to parents. In addition to location and contact information, a small set of additional items will provide information on the parents’ perception of the children’s social and emotional skills and behavioral outcomes.
The tracking and follow-up surveys will primarily be conducted over the telephone with in-person follow-up as necessary. These surveys will take about 30 minutes to complete.
We will use two measures in the parent survey that have previously been used in the Head Start CARES project: the Social Skills Rating System (SSRS; Gresham & Elliott, 1990) and the Behavior Problems Index (BPI; Zill & Peterson, 1986). These measures focus on both adaptive and problem behaviors, key foci and targets of the Head Start CARES enhancements. Additionally, they were both used in previous waves of data collection for Head Start CARES (preschool and kindergarten), and therefore data collected from primary caregivers in tracking children to first, second, and third grade will be aligned with data collected in the preschool and kindergarten year. The SSRS Preschool Level questionnaire (SSRS; Gresham & Elliott, 1990) will be replaced by the SSRS Elementary Level questionnaire. The Tracking and Follow-Up Parent Survey is included in Appendix A. Specific changes made to the Kindergarten Follow-Up Parent Survey from the previous OMB approved Head Start CARES extension in the development of the Tracking and Follow-Up Parent Survey for the current study are shown below in Exhibit A.2.
Exhibit A.2. Changes to the Previous OMB Approved Kindergarten Follow-Up Parent Survey for the Current Tracking and Follow-Up Parent Survey
Specific Question(s) or Questionnaire |
Type of Change |
Source of Changes |
Tracking Script and Contact Information |
||
Questions A1 ― A10* |
Added |
Tracking Head Start Impact Study 8th Grade |
Child’s School Experiences |
||
Questions B1 ― B7 |
No Change |
― |
B8. During the past 12 months, has the school contacted you or another adult in your household about any problems [CHILD] is having with school, including academic concerns and/or behavior problems? |
Added |
Adapted from the National Survey of Children’s Health (NCHS)1 |
B9. During the past 12 months, has the school contacted you or another adult in your household about [CHILD] for positive reasons, including academic success and/or improvements in behavior? |
Added |
Adapted from NCHS |
B10. During the past 12 months, has [CHILD] received… a. Speech or language therapy? b. Occupational therapy? Occupational therapy helps with the strengthening of fine motor skills including writing, using utensils, cutting, and tying shoe laces. c. Treatment or counseling from a mental health professional? Mental health professionals include school counselors, psychologists, psychiatrists, psychiatric nurses, and clinical social workers. d. Physical therapy? e. Special instruction or tutoring? |
Added |
Adapted from NCHS |
B11. Since starting kindergarten, has [CHILD] repeated any grades? |
Added |
Adapted from NCHS |
B12. Which grade or grades did [CHILD] repeat? |
Added |
Adapted from NCHS |
Child’s Social and Emotional Skills |
||
Questions C1 ― C38: Social Skills Rating System (SSRS) |
New Version
|
Elementary Level version of SSRS |
Child’s Behavior Problems |
||
Questions D1 ― D29 Behavior Problem Index (BPI) |
No Change |
― |
Other Parent Assessments |
||
Parent-Teacher Involvement Questionnaire |
Omitted |
― |
Parent Stress Index (PSI) |
Omitted |
― |
Economic Outcome Questions included in the Foundations of Learning |
Omitted |
― |
*Note: Specific survey questions referenced by the letter and number can be found in Appendix A. Tracking and Follow-Up Parent Survey.
A3. Improved Information Technology to Reduce Burden
The tracking interviews will be conducted primarily over the telephone, with in-person interviews as necessary. The use of Computer Assisted Telephone Interviewing (CATI) has been incorporated into the data collection of the parent surveys in order to ensure accuracy of data, reduce possibility for human error, allow for faster data analysis and reduce respondent burden. Other non-technology efforts to reduce burden include training interviewers extensively and sections in the survey with lead questions to enable skip patterns.
A4. Efforts to Identify Duplication
The Head Start CARES project is the first investigation to use a nationally representative sample of Head Start programs to examine new and promising approaches for enhancing social and emotional development in the preschool years. The impact and implementation study provided the opportunity to look at effects into the kindergarten year. This continued tracking will ensure sufficient maintenance of the original Head Start CARES project to allow children’s longer-term follow-up during or after the third grade. The current surveys focus on information that cannot be found in administrative records or other existing sources. These surveys will facilitate the collection of data on, for example, children’s social and emotional skills and behavioral outcomes.
A5. Involvement of Small Organizations
No small businesses or other small entities will be involved in the data collection for this tracking study.
A6. Consequences of Less Frequent Data Collection
Longer periods between data collections would risk lower response rates with respondent interest possibly waning and family movement would be more difficult to track. The early school years are a time where children’s social and emotional skills are rapidly developing. Therefore, infrequent data collection on children’s social and emotional skills and behavioral outcomes would increase the risk of not adequately capturing the impact of the Head Start CARES enhancements.
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on Friday, August 17, 2012, Volume 77, Number 160, page 49817, and provided a sixty-day period for public comment. A copy of this notice is attached as Appendix B.1. During the notice and comment period, 0 comment(s) were received.
We have developed instruments that incorporate items and scales from other major studies. To the extent possible, the questions included in the survey instruments allow for useful comparisons between the data from this project and that from other large-scale surveys. To select these measures for the various components of the survey instruments and implementation measures, we consulted throughout the study with a number of individuals outside MDRC, including: Cybele Raver, Clancy Blair, Catherine Tamis-LeMonda (New York University); Karen Bierman, Robert Nix, Mark Greenberg, Celene Domitrovich (Pennsylvania State University); Nancy Hill, Stephanie Jones, Hirokazu Yoshikawa (Harvard University); Mary Louise Hemmeter (Vanderbilt University); Todd Little (University of Kansas); Nicholas Ialongo (Johns Hopkins University); Susanne Denham (George Mason University); John Lochman (University of Alabama); George Knight (Arizona State University); Bob Pianta and Bridget Hamre (University of Virginia); Dwayne Simpson (Texas Christian University); Julie Hakim-Larson (University of Windsor); Deborah Leong (Metropolitan State College of Denver); Carolyn Webster-Stratton (University of Washington); Allison Sidle Fuligni, Carollee Howes, Sharon Ritchie (UCLA); Gary Henry (University of North Carolina at Chapel Hill); Douglas Powell (Purdue University).
A9. Incentives for Respondents
We recognize that participation in the current Head Start CARES surveys will place some burden on the participating parents. Although many of the techniques suggested by OMB to improve response rates have been incorporated into our carefully designed instruments and the survey effort (described in Section B.3), it has been our experience that small tokens of appreciation are useful when surveying teachers and low-income populations as part of a complex study design in order to acknowledge the burden placed on participants.
Incentives are important, especially in a longitudinal study, to gain respondents’ cooperation and ensure a high response rate and their participation throughout the study, both at the baseline and at the follow up interview (e.g., James, 1997; Mack et al., 1998). Incentives are most appropriately used in federal statistical surveys with hard-to-find populations or respondents whose failure to participate would jeopardize the quality of the survey data (e.g., in panel surveys experiencing high attrition), or in studies that impose exceptional burden on respondents, such as those asking highly sensitive questions.
We have based the amount of the token of appreciation offered to respondents on prior research and on MDRC’s and SRM’s prior experience interviewing similar populations. We propose that the token of appreciation be $20 for the 30 minute parent survey. These amounts reflect current practice in surveys using similar instruments and will be provided in the form of a gift card.
The parent survey token of appreciation is comparable to amounts used in FACES ($35 for 45 to 60 minute interview); Baby FACES ($35 for a 120 minute interview); and Building Strong Families (BSF; $50 for completing two 50 minute parent interviews). The amount is sufficient to encourage families to participate in both the study and the survey but is not overly generous. Offering a lower amount could jeopardize the study and actually cost the government more because it could result in a lower uptake of families into the study and more effort expended by the evaluation team to successfully find families.
Privacy will be assured to the fullest extent allowable under the law. Respondents will receive information about privacy protections at the outset of the surveys. They will be informed that all of the information they provide will be kept strictly private and that study results will be presented only in aggregate form. They will also be told that completion of the survey is voluntary and that they may choose not to answer any question.
The following safeguards will be employed regarding privacy assurances:
All staff who have access to data at MDRC and SRM sign an agreement to abide by corporate policies on data security and privacy. This agreement affirms each individual's understanding of the importance of maintaining data security and privacy and abiding by procedures that implement these policies.
All data, both paper files and computerized files, are kept in secure areas. Paper files are stored in locked storage areas with limited access on a need-to-know basis. Computerized files are managed via password control systems to restrict access as well as physically secure the source files.
Merged data sources have identification data stripped from the individual records or encoded to preclude identification of individuals.
All reports, tables, and printed materials present aggregate numbers only.
Compilations of individualized data are not provided to participating agencies.
Agreements are executed with any participating research subcontractors, partners, and consultants who obtain access to data files.
MDRC and SRM will maintain in-house records of names, addresses, school identification numbers (if applicable), and tracing information for all sample members. This information will not be attached to survey or assessment data or made available to anyone outside appropriate staff of MDRC and SRM. All records identifying respondents will be kept in locked storage at MDRC, and respondents will be identified solely by a code number. Any coding, data entry and analysis requiring identification of individuals or households will use code numbers only, and a secret password will be necessary to access the data file. No data will ever be reported in such a way that individuals can be identified. In developing the public use file, we will be implementing data masking procedures to ensure that sample members cannot be identified individually. See Appendix C for an example of procedures that were developed for another Department of Health and Human Services project conducted by MDRC. We will implement similar masking procedures for this project.
The importance of maintaining privacy will be emphasized during interviewer training, and any interviewer who knows a respondent will not be permitted to interview him or her. All staff, including coders and computer programmers, will be required to sign a privacy pledge.
At the beginning of each interview, respondents will be informed of their rights. In addition, interviewers will attempt to conduct the interview at a time and place that allows the utmost privacy for respondents over the phone.
We do not anticipate that any of the questions asked will be of a sensitive nature. The purpose of the interview and how the data will be used will be explained to all participants.
A12. Estimation of Information Collection Burden
Previously Approved Information Collections
Total Burden Previously Approved
For the previous Kindergarten collection, 401.28 burden hours were approved.
Burden Remaining from Previously Approved Information Collection
All data collection from the previously approved data collection listed in Section A.1 will be completed by the time the information collection request is approved.
Newly Requested Information Collections
Exhibit A.3 presents the estimated annual burden for respondents to the new Head Start CARES tracking and brief follow-up study for each data collection points. The estimated response burden by instrument/component was calculated based on information on survey length obtained during the pretests (see Section B.4).
The total number of respondents (3,633 parents and guardians) is in the table by cohort and age: Cohort 1 (4-year-olds); Cohort 2-(4-year-olds); and Cohort 2 (3-year-olds). All respondents will respond to the same survey (see Appendix A). The total number of respondents in these groups were first multiplied by the annual number of responses per respondent, and then multiplied by the average burden hours per response to get the total burden hours per cohort. Since the project is over a 3-year period of time, the total burden hours were divided by 3 to determine the annual burden hours. The estimated annual burden sub-total was the sum of these total burden hours across cohort-ages and the sum of the annual burden hours across cohort-ages.
Exhibit A.3: Annual Information Collection Burden and Cost
Instrument |
Total Number of Respondents |
Number of Responses Per Respondent |
Average Burden Hours Per Response |
Total Burden Hours |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Parent Survey Cohort 1 (4-year-olds) |
603 |
2 |
0.50 |
603
|
201
|
$15.03 |
$3021.03 |
Parent Survey Cohort 2 (4-year-olds) |
2070 |
3 |
0.50 |
3105 |
1035 |
$15.03 |
$15556.05 |
Parent Survey Cohort 2 (3-year-olds) |
960 |
4 |
0.50 |
1920 |
640 |
$15.03 |
$9619.20 |
Estimated Annual Burden Sub-total |
5,628 |
1,876 |
|
$28,196.28 |
|||
Total Annual Cost
To compute the total estimated annual cost, the total burden hours were multiplied by the average hourly wage for the labor category. For parents, we used the mean salary for full-time employees over the age of 25 who were high school graduates with no college experience ($15.03/hour). The total estimated annual cost is $28,196.28.
A13. Cost Burden to Respondents or Record Keepers
There are no direct monetary costs to participants other than their time to participate in the study.
A14. Estimate of Cost to the Federal Government
The total cost for the data collection activities under this current request will be $1,039,398. These costs include such activities as development of the tracking materials, data collection, and preparation of a data file. Annual costs to the Federal government will be $346,466 for the proposed data collection.
This is an additional information collection request under this OMB number.
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication
A16.1a Assessment of Data Quality and File Construction
These surveys have gone through a rigorous series of tests for completeness and quality. Professional staff at the survey firm, SRM, will review the initial cases completed by each interviewer as well as perform occasional spot checks after that. Interviewers will be apprised of any problems found and retrained if needed. The survey firm will deliver data sets of completed cases at agreed-upon internals, along with marginal frequencies. The data and frequencies will be reviewed for outliers, unusual distributions and inconsistencies between data items.
A16.1b Impact Data Analysis
The net impacts of each intervention will be measured by comparing measures of outcomes for students for each treatment group to those for the control group.
Data reduction. We will use existing approaches developed in developmental psychology for data reduction of our individual survey items into scales representing our constructs of interest. Data reduction techniques used in previous waves of the data collection will be employed in the current tracking and follow-up study. For example, we will examine inter-item correlations for the full set of questions designed to measure this outcome and conduct a factor analysis where appropriate to determine that items in the set “go together” as they did in previous waves and appear to be measuring the same underlying construct. Next, we will estimate Cronbach's alpha to assess the reliability of the scale. After selecting the final set of items for a given scale, we will then produce an overall scale score for each respondent by summing his/her scores on each of the items in the scale. The overall scale scores for all respondents will then be used as an outcome measure for the impact analysis. We have used this general approach successfully in several previous evaluations, especially the more recent evaluations with child outcomes data.
Impact analysis. Our impact analysis will focus on the net impacts of each intervention on students. Net impacts will be estimated by comparing mean outcomes for each intervention group to corresponding means for the control group with a regression-adjustment for selected background characteristics. Wherever possible the adjustment will control for a baseline measure of the outcome (a “pretest”), because it is usually the most powerful predictor of future outcomes and thereby typically provides the biggest boost possible to statistical precision (or power). These analyses compare a single intervention to the control group. They will be conducted for each intervention tested.
A16.2 Publication Plans and Schedule
Data and findings will be issued and shared following the completion of the Cohort 2 three-year- old data collection in 2016.
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
1 Aber, Jones, and Cohen (2000); Brooks-Gunn, Klebanov, Liaw and Spiker (1993).
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |