NYTD ICR Supporting Statement-102909 FINAL

THE SUPPORTING STATEMENT

Specific Instructions

Please do not remove or alter the headings below

A. Justification

1. Circumstances Making the Collection of Information Necessary

Public Law 106-169, enacted December 14, 1999, established the John H. Chafee Foster Care Independence Program (CFCIP) at section 477 of the Social Security Act (the Act). The law provides States with flexible funding to carry out programs that assist young people in making the transition from foster care to self-sufficiency. The law requires the Administration for Children and Families (ACF) to develop a data collection system to track the independent living services States provide to youth and develop outcome measures that may be used to assess States’ performance in operating their independent living programs. We are implementing this requirement by creating a collection of information entitled the National Youth in Transition Database (NYTD). The regulations for the NYTD are found at 45 CFR 1356.80. There are two information collection instruments associated with this request: the Data File and the Youth Outcome Survey.

2. Purpose and Use of the Information Collection

States will report information about certain youth both in and out of foster care and the services that are traditionally offered to young adults in preparation for emancipation. States will be able to use the information to improve their independent living programs to assist youth transition to independence. We will use this information to track independent living services, to assess the collective outcomes of youth, and to evaluate State performance with regard to those outcomes consistent with the law’s mandate. While OMB approved the information collection request pertaining to NYTD for an 18-month period (expiring 11/30/2009), no information has been collected yet. Pursuant to the regulation, States will begin collecting NYTD information on October 1, 2010 and will submit the first semi-annual data file to ACF no later than May 15, 2011.

3. Use of Improved Information Technology and Burden Reduction

States will submit NYTD data to the Department in an electronic format. Although ACF has not released final technical specifications for the transmission of NYTD data files, we have decided to use Extensible Markup Language (XML) for file transmission consistent with the E-Government Act of 2002 (Public Law 107-347) based, in part, on public responses we received during the NYTD rule-making process. A draft technical bulletin that describes the proposed XML format that will be used to submit NYTD data was distributed to States in May 2008. We are in the process of finalizing this technical bulletin based on State feedback.

4. Efforts to Identify Duplication and Use of Similar Information

Congress specifically mandated that we collect data on independent living services and youth outcomes. We have analyzed the sources and reporting instruments already in use in the collection of independent living services information and have concluded that the type of data in the NYTD is not collected elsewhere. Specifically, we examined both Federal and non-Federal data sources such as:

• Federal data collection systems such as the Adoption and Foster Care Analysis and Reporting System (AFCARS), and the Runaway and Homeless Youth Management Information System (RHYMIS);

• State, county, and local governments with integrated and/or complementary data systems such as Statewide Automated Child Welfare Information System (SACWIS); and

• Data collection efforts of non-governmental organizations such as Casey Family Programs, United Way, Lutheran Social Services, Catholic Social Services, Child Welfare League of America, and Public/Private Ventures.

AFCARS collects data on youth that are in foster care or that were adopted under the auspices of State child welfare agencies. Many youth that will be reported by States to the NYTD are also going to be reported to AFCARS. However, youth receiving independent living services may not be in foster care. Youth in the follow-up population will generally not be in foster care at ages 19 and 21. Finally, there are separate and different authorizing statutes and penalty structures for NYTD and AFCARS that do not lend themselves to combining the databases.

5. Impact on Small Businesses or Other Small Entities

This information collection is required of State agencies only and does not impact small businesses or other small entities.

6. Consequences of Collecting the Information Less Frequently

This data collection is mandated by law, therefore we would be out of compliance with the statutory requirements if we did not collect independent living service and youth outcome information. In the regulation, we require that States submit NYTD data to ACF every six months, which is the same reporting frequency as AFCARS. We believe that any less frequent reporting may increase the risk of States reporting inaccurate or missing data. Further, we chose a semi-annual reporting period to preserve our ability to analyze NYTD data along with AFCARS data for the same youth. The six-month report period for AFCARS is integral to a number of ACF priorities and legislative requirements. 

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances required in the collection of this information in a manner other than that required by OMB.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In response to the Proposed Information Collection Activity Comment Request, published in the Federal Register on August 5, 2009 (74 FR 39077), we received nine public comments. Most commenters offered at least one recommendation related to changing the implementation date for NYTD, changing the NYTD data element definitions, or changing the definitions of the NYTD reporting populations. Because these parameters were established in regulation as part of our rule-making, these recommendations are beyond the scope of consideration for this information collection activity comment request.

Two commenters disagreed with our burden estimates on States for preparing the NYTD Data File explaining that necessary administrative changes such as SACWIS enhancements and related training for caseworkers as well as outreach to contracted service providers in order to collect information on services they provide to the NYTD served population will require time that is not accounted for in our burden estimates. We are not changing our burden estimates related to States collecting and reporting information on youth independent living services as part of the NYTD Data File as suggested by commenters. It is our expectation that States will implement appropriate child welfare information system enhancements prior to FY 2011 to capture adequately the independent living services paid for or provided to youth by the State.

All nine commenters disagreed with our burden estimates related to the NYTD Youth Outcome Survey, explaining that the estimate (0.17 burden hours per response) likely did not take into account the amount of time necessary to locate youth, explain the purpose of the Youth Outcome Survey, clarify the content of questions that may be confusing, and compile all the survey information to transmit as a data file. It is likely that commenters were confused by our averaging the annual burden hours associated with a youth taking the NYTD Youth Outcome Survey (0.5 hours) over a three period (FY 2010 – FY 2012) when survey data is only collected one of these years (FY 2011). Also, while we did factor in burden hours associated with a State tracking youth and compiling survey information for transmission in the State’s Data File, we applied these burden hours to the Data File information collection instrument and not the Youth Outcome Survey instrument. Further, because youth in the baseline population will be in the State’s placement and care responsibility, we did not initially allocate burden hours to tracking youth in FY 2011 and instead allocated them to FY 2012—the year in which States will likely begin implementing protocols for keeping in touch with youth as they leave care.

We agree with the commenters, however, that our initial burden estimates related to administering the Youth Outcome Survey were low. As we continue to discuss NYTD implementation with States and provide technical assistance, we now agree that many youth will need more than 30 minutes to complete the 22-question survey, particularly youth who have developmental delays or youth for whom English is not a first language. While States will select different approaches to administering the survey, we feel that a more accurate estimate of burden is 45 minutes for a youth to take the survey. We also are amending our estimate of burden hours associated with a State’s collection of youth outcomes information from 15 minutes to 30 minutes on average to ensure States are able to explain the purpose of the survey appropriately to the youth and to ensure all questions and response options are clear to the youth. Many of the commenters also expressed concerned about the quality and utility of the information collected as part of the NYTD Youth Outcome Survey. Four commenters, for example, questioned the reliability of survey data that is self-reported by youth. Because several survey questions ask youth about high-risk behaviors or ask youth about services a youth may not be aware of having received, these commenters suggested that youth responses may not be truthful or accurate. Two commenters expressed concern about the impact of non-response bias on the quality of survey data when youth with certain characteristics may be disproportionately represented among the total youth the State engages to participate in the survey. One commenter, for example, suggested it may be easiest to track youth who are incarcerated or those who use public assistance and consequently the survey data will overrepresent the negative experiences of youth who engage in criminal behaviors or who experience poverty. Another commenter suggested that youth who remain connected to the agency through the State’s Independent Living Program after age 18 will be overrepresented in the survey data because it will be simple for the State to locate and engage these youth in the survey. (It should be noted that surveying of the first follow-up population cohort will not begin until FY 2013, which is beyond the three-year information collection approval we are requesting at this time).

The goal of the Youth Outcome Survey is to collect information that represents the youth's understanding of his or her own circumstances and experiences as a whole. To achieve this goal and to preserve the integrity of the survey and data, we specified in the regulation that States must administer the survey to youth directly and may not provide information on outcomes data elements from any other source including administrative databases, such as Temporary Aid for Needy Families (TANF), Social Security or employment records (73 FR 10350). Because States will be required to survey a population of young adults that are historically difficult to track and because the survey may be administered in a format selected by the State, we acknowledge the potential for non-response bias in NYTD survey data. Through technical assistance with States, we have addressed methods to minimize non-response bias and maximize youth participation. These technical assistance efforts are highlighted in greater detail in our response to item B.3 of this supporting statement as well as in our current written guidance to States on this subject in the attachment Practical Strategies for Planning and Conducting the National Youth in Transition Database (NYTD) Youth Outcome Survey. We feel our continued efforts to address non-response bias through technical assistance and data analysis is sufficient to minimize threats to survey data quality.

Commenters also recommended strategies for minimizing burden on State and youth respondents. Four commenters indicated interest in developing web-based applications to collect youth survey data efficiently. Some of these commenters, however, expressed concerns about whether such an application would be compliant with SACWIS requirements, would address State data security concerns, or would be affordable. Three commenters recommended exploring the use of social networking technologies like Facebook to ease the burden of keeping in touch with youth after they leave foster care. We agree that web-based technologies have great potential to assist States in locating and engaging youth and young adults in the NYTD Youth Outcome Survey in an efficient and cost-effective way. While we have not required a specific method States must use to collect the survey data or track youth that leave foster care, we continue to provide technical assistance and other guidance to States on appropriate and allowable methods for locating youth and administering the survey (See attachment Practical Strategies for Planning and Conducting the National Youth in Transition Database (NYTD) Youth Outcome Survey).

Two commenters suggested that providing incentives to youth respondents may boost participation in the Youth Outcome Survey. We agree that incentives, if administered properly, encourage and reward participation in a data collection effort like NYTD, particularly among hard-to-find populations like emancipated youth whose failure to participate could impact the quality of NYTD survey data. We have encouraged States to be mindful of guidance issued by the OMB entitled "Guidance on Agency Survey and Statistical Information Collections." While this guidance is directed to Federal agencies, the same general principles should guide State agencies as well.

9. Explanation of Any Payment or Gift to Respondents

No payments, other than Federal financial participation, will be made to States for the maintenance and development of an information system.

10. Assurance of Confidentiality Provided to Respondents

The final rule requires States to use an encrypted personal identification number so the identity of the individual youth remains confidential to anyone other than the State.

11. Justification for Sensitive Questions

Congress specifically required in law that States collect outcome information on issues of a sensitive nature including rates of non-marital childbirth, incarceration and high risk behaviors. Therefore, we pose questions in the youth survey that ask about these areas of a youth’s life. The survey is voluntary and we expect States to obtain youth or parental consent, as appropriate.

12. Estimates of Annualized Burden Hours and Costs

This information collection will be comprised of:

1. Data File: The State’s submission to ACF of two semi-annual data files that contain information on all data elements regarding youth services, demographics, characteristics and outcomes. A State will collect this information on an ongoing basis. The total annual burden will vary from year to year; the burden will be lower in years in which States do not have to collect information on youth outcomes. Years in which a State must expend effort to track or maintain contacts with youth as they age from 17 years to 21 years will have the highest total burden hours; and,

2. Youth Outcome Survey. A survey composed of up to 22 questions on youth outcomes (that correspond with 22 data elements in the first instrument) to be completed by youth in the baseline and/or follow-up populations.

Annual Burden Hour Estimates for the NYTD, FY 2010 – FY 2012

The following are burden hour estimates for FY2010, FY2011, and FY2012. Please note that the average annualized burden estimates (covering all three years of this information collection request) are detailed in the table labeled “Annualized burden estimates for the NYTD for the 3-year period corresponding to this ICR, FY 2010 – FY 2012” on page 7 of this document.

1. FY 2010: States will not have to collect any information with regard to the two instruments in FY 2010, thus there is no hour burden associated with the information collection in accordance with the PRA. However, States will have to change their internal business procedures and information systems to meet the information collection requirements in subsequent years.

FY2010

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	0	0	0	0
Services Outcomes Tracking			0	0
			0	0
			0	0
2. Youth Outcome Survey	0	0	0	0
TOTAL burden for both collections				0

2. FY 2011: With regard to the data file instrument, we estimate that there will be approximately 2,518 youth who receive services annually per State. Each State will expend on average 30 minutes (0.5 hours) to collect the services, demographics and characteristics information from those youth resulting in an hour burden of 1,259 per State each report period. FY 2011 will be the first year of outcomes data collection for the baseline population of 17-year-olds in foster care. We are estimating 30 minutes (0.5 hours) for States to explain the purpose of the survey to the youth and to record the outcomes information that will be reported in the data file. We estimate that there will be on average 766 17-year-old youth in foster care per State in FY 2011 resulting in approximately 192 burden hours to record this information each report period (assuming that States survey half of the baseline population of 17-year-olds in each of the two semi-annual reporting periods in FY 2011). We are not estimating tracking burden during this year as we expect States to know the whereabouts of the 17-years-olds still in foster care. The total hour burden for the data file instrument is 150,904. With regard to the outcome survey instrument, we are estimating that it will take the estimated 39,832 youth nationwide approximately 45 minutes (0.75 hours) to complete the outcomes survey, resulting in a total of 29,874 burden hours.

FFY 2011

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	52	2	1,451	150,904
Services Outcomes Tracking			1,259.0	130,936
			192*	19,968
			0	0
2. Youth Outcome Survey	39,832	1	0.75	29,874
*Number is rounded TOTAL burden for both collections				180,778

3) FY 2012: With regard to the data file instrument, we have estimated the same 1,259 hour burden per State per report period associated with collecting and reporting youth independent living services, demographics and characteristics information. In FY 2012, however, there will be no outcomes data collection due to us so there is no burden associated with such reporting. However, States will have to track youth who have participated in the baseline population at age 17 so that they can solicit their participation in outcomes data collection when such youth turn 19 years old. Therefore, we have estimated that each State will spend approximately 30 minutes (0.5 hours) per youth per report period to track approximately 766 youth, resulting in a total burden of 383 burden hours per State. There is no outcomes data collection in FY 2012, so there is no hour burden associated with the youth outcome survey instrument.

FFY 2012

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	52	2	1,642	170,768
Services Outcomes Tracking			1,259.0	130,936
			0	0
			383	39,832
2. Youth Outcome Survey	0	0	0	0
TOTAL burden for both collections				170,768

Annualized burden estimates for the NYTD for the 3-year period corresponding to this ICR, FY 2010 – 2012

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	35	2	1,031	72,170
2. Youth Outcome Survey	13,278	1	0.25	3,320
TOTAL annualized burden estimate for both collections				75,490

Annualized Cost Estimates for the NYTD, FY 2010 – FY 2012

Using $25.00 per hour as an approximation for State worker hourly wages, the costs associated with gathering the Youth Outcome Survey information and preparing and submitting the NYTD Data File would be $72,550 per State for FY 2011 (1,451 burden hours per response x 2 responses per year x $25.00 hourly wage) and $82,100 per State for FY 2012 (1,642 burden hours per response x 2 responses per year x $25.00 hourly wage). Because no data is required to be collected or reported to the NYTD in FY 2010, we are not estimating costs to respondents during this year. We are not estimating any costs to the youth who participate in the NYTD Youth Outcome Survey in FY 2011.

Annualized cost estimates for the NYTD for the 3-year period corresponding to this ICR, FY 2010 – 2012

Instruments (subcomponents)	Total cost
1. Data File	$51,550*
FY 2010	$0
FY 2011	$72,550
FY 2012	$82,100
*Or $25,775 per response as indicated in the ICRAS Jr, Item 11. There are two responses per year.
2. Youth Outcome Survey	$0
FY 2010	$0
FY 2011	$0
FY 2012	$0

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

We expect that on average business process start-up costs, travel and training, process development, information technology (IT) start-up operational and maintenance costs will be approximately $250,000 per State per year. Because many of these costs can be matched at 50% Federal financial participation for a State that has a SACWIS, we are estimating a cost burden of $125,000 per State per year (or $62,500 for each of the two responses per year). We do not expect that reporting for NYTD will require the purchase of additional hardware or other capital equipment. For IT changes, the main costs are in software development and/or modification. We characterize these costs as minor, but we are aware of the impact the cost of implementation will have on States. During the rule-making process, we received comments from States regarding anticipated start-up and annual costs that indicated our estimates are accurate. We have applied these estimates to the cost burden calculation on item 11 of the ICRAS Jr. Part II for the NYTD Data File under “reporting.” We are not estimating any other cost burden associated with the Youth Outcome Survey instrument because we are not estimating any costs to respondent youth and because costs associated with a State collecting survey data is captured as part of other IT system development costs in the Data File instrument (as shown below).

Estimates of other total annual cost burden for reporting for the 3-year period corresponding to this ICR, FY 2010 – FY 2012

Instruments (subcomponents)	Cost per response	Responses per year	Number of respondents	Total cost burden for reporting
1. Data File	$62,500*	2	35	$4,375,000
2. Youth Outcome Survey	$0	1	13,278	$0

*$250,000 estimate per year, matched at 50% Federal financial participation = $125,000 per year or $62,500 for each response per year

We recognize that the reporting and recordkeeping burden is disproportionately higher for small States because they need to develop the same functionality as large States regardless of the number of youth reported. State costs are not affected by the number of youth in the reporting population or the number of reporting periods. We also recognize that county-administered States may face more challenges in implementation; but saw no need to change the rule in response. Each State will have different costs depending on their populations of youth and the changes required in their information systems and business procedures to meet the requirements in the regulation.

14. Annualized Cost to the Federal Government

The costs for a contractor to design and implement a Federal computer system to receive and analyze State data, provide training and technical assistance (TA) to both State and Federal users on the NYTD system, and to host a national TA meeting each year are $1,924,315 for FY 2010, $1,204,229 for FY 2011, and $830,711 for FY 2012.

15. Explanation for Program Changes or Adjustments

The previous information collection approved by OMB applied only to May 2008 – November 2009, a period in which no data was scheduled to be collected.  Consequently, all burden hours, number of respondents and cost burden figures that appear in this request are new because States will begin collecting and reporting NYTD data to ACF pursuant to the regulation in FY 2011 for the first time.  This is defined as a program change as this information collection activity is the result of a regulation published in February 2008 that mandates data collection during the three-year approval period of this ICR (see attached NYTD regulation). We are not proposing any other material changes in the collection instruments, instructions, or the uses of the information collection currently approved by OMB. 

16. Plans for Tabulation and Publication and Project Time Schedule

The regulation requires States to begin collecting data as of October 1, 2010 and report information to ACF as of May 15, 2011. We plan to make some data reports available on the Children’s Bureau website. We cannot finalize a schedule for analyzing and publishing the data until after we receive State data.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The Department is requesting that the OMB number and expiration date not be displayed for the NYTD Data File as these files will be transmitted electronically to ACF. The Department is also requesting that the OMB number and expiration date not be displayed on the NYTD Youth Outcome Survey. While we chose to regulate the survey questions and response options in the Youth Outcome Survey, there is no specified form or format in which States must administer the survey to youth. Instead, the Department requests permission to display the OMB number and expiration date in the regulation by technical amendment.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

B. Statistical Methods (used for collection of information employing statistical methods)

1. Respondent Universe and Sampling Methods

The NYTD Data File instrument collects semi-annual information on all NYTD data elements regarding youth services, demographics, characteristics and outcomes. Fifty-two respondent States will collect this information on an ongoing basis. No statistical methods are used or required for this instrument other those used for the Youth Outcome Survey, which is a component of the Data File.

The potential respondent universe for the Youth Outcome Survey instrument consists of 17-year-olds who are in State foster care systems during a Federal fiscal year, beginning in FY 2011, with a new cohort selected every three years thereafter. States will get this information from data submissions submitted to the AFCARS. The respondent universe includes foster care youth who receive independent living services and those that do not and is defined in 45 CFR 1356.81(b) and 1356.82(a)(2). Youth that are incarcerated or institutionalized in a psychiatric facility or hospital would not be a part of the baseline population because they are not in foster care consistent with the definition found in 45 CFR 1355.20. According to data from AFCARS, approximately 40,000 youth in foster care at the end of fiscal year 2006 met the criteria for inclusion in the respondent universe.

Depending on the number of actual baseline respondents in a State, the State may choose to sample respondents for the follow-up population. The sampling formula is regulated in 45 CFR 1356.84. The sampling universe consists of youth in the State in the baseline population who participated in the State’s agency’s data collection at age 17. A State that chooses to sample must use simple random sampling procedures based on random numbers generated by a computer program unless another accepted methodology is approved by ACF. States that choose to sample will conduct sampling following the baseline year data collection. States report to ACF in their data file the identifiers of those youth in the sample.

Based on our understanding of the population and similar efforts to obtain outcomes data on this population, we estimate that response rates for the baseline population will be 90% of 17-year-olds, 80% of 19-year-olds and 60% of 21-year-olds. To comply with the NYTD, States must achieve response rates of 60% of the 19- or 21-year olds who have aged out of foster care.

2. Procedures for the Collection of Information

The NYTD Data File instrument collects semi-annual information on all NYTD data elements regarding youth services, demographics, characteristics and outcomes. No statistical methods are used or required for this instrument other than those used for the Youth Outcome Survey, which is a component of the Data File.

As stated in the response to B.1, States will conduct the Youth Outcome Survey on a three year wave basis, starting with a new universe of 17-year-olds every three years. After States establish their first baseline population cohort in FY 2011, States that choose to sample will employ simple random sampling or they may request ACF approval of another accepted sampling methodology. ACF will not accept proposals for non-probability sampling methodologies, but will consider stratified random sampling and other probability samples that generate reliable State estimates. The sampling universe will consist of the total number of youth in the baseline population that participated in the data collection at age 17.

States will administer to youth the survey located in appendix B to the regulation. States have the discretion to conduct the surveys via in-person interviewers, computer-aided devices, phone interviews or other methods as it suits their particular needs and population. There are no dedicated resources under 42 USC 677 for States to devote to this data collection effort and funds will likely come from a combination of funds that would otherwise be used for youth independent living services and other existing resources. Given these limited State resources, and our need for data primarily as an administrative database and oversight tool, we declined to prescribe a particular survey method as is commonly used in research practices. Through technical assistance, ACF will also encourage States to use methods that are likely to achieve high response rates. We anticipate that most States will use in-person interviews or a combination of in-person and computer-aided techniques given the recent findings from studies such as the Multi-Site Evaluation of Foster Youth Programs that indicated high response rates with these approaches (U.S. Department of Health and Human Services, 2007).

Through conference calls, site visits, annual meetings and written publications, ACF has provided technical assistance to States to encourage best practices in tracking youth and administering the survey regardless of the method chosen. Attached to this request package is ACF’s current guidance to States on administering the survey (See Practical Strategies for Planning and Conducting the National Youth in Transition Database (NYTD) Youth Outcome Survey). Technical assistance on sampling will be conducted primarily with ACF statisticians, while assistance with tracking youth and administering the survey will be provided through our technical assistance partners, the National Child Welfare Resource Center for Youth Development and the National Resource Center for Child Welfare Data and Technology.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Our expected response rates are modeled on RR2 response rate (American Association for Public Opinion Research, 2006) and are reflective of our analysis of information from data collection efforts on former foster youth sponsored by ACF and States. As we indicate above, we anticipate a 90% response rate from our baseline population of 17-year-old youth in foster care. While this population is easy to locate because they are in the placement and care responsibility of a State agency, we anticipate that some youth will not be able to be located because they have run away from foster care. In addition, a small percentage of youth may decline to participate in the survey. We believe that our anticipated response rate is suitable for our purposes, which is to have some outcomes information to meet the statutory mandates at 42 USC 677 that can provide a perspective on how youth are faring as they prepare to leave foster care and assess State performance of their independent living programs. For the baseline population of 17-year-olds the response rate numerator is the number of 17-year-old foster youth who complete or partially complete the outcomes survey. The response rate denominator is the total number of 17-year-old youth in foster care minus youth who are not eligible to be interviewed because they are mentally or physically incapacitated, deceased or incarcerated. There is no participation rate standard States must meet for youth in the baseline population in the NYTD regulation.

Approaches to collecting the Youth Outcome Survey data are expected to vary as States select the most appropriate approach to meet the needs of the State and the particular characteristics of the State's population (collection of data in-person, by telephone, using computer-aided devices, etc.). While each approach to data collection has the potential for non-response bias, response bias, and measurement error, there also are standards of practice for collecting data to address potential bias, and the Children's Bureau is well-prepared to address this issue. Because it is expected that many States will choose to administer the survey during personal interviews, an important training area will be that of interviewing techniques. These techniques will need to be tailored to account for the individual characteristics and needs of the respondents. For example, some respondents may have disabilities, such as limited vision or hearing, and the TA providers will need to give States advice and information on how to address such interviewing challenges so that the respondents feel comfortable enough to proceed with the survey and provide accurate responses.

Survey researchers can and do use information on differential response rates to create weights that are used to correct bias in the data due to non-responders (Holt and Elliot, 1991), (Nathan Berg, 2002). After the Children's Bureau receives the NYTD data from the States, these corrections can be done at the Federal level using standard methods. We intend to conduct non-response analysis using data available from NYTD on demographic characteristics and independent living service history and AFCARS data regarding disabling conditions, length of time in foster care, foster care placement setting history and other factors.

Measurement error also can occur due to the respondent's inability to understand certain questions. Because of the likelihood of a wide range of comprehension levels in the target population, the TA staff will provide States with advice in dealing with this issue to ensure the most accurate collection of the information from the target youth. We will continue to provide technical assistance, for example, on the use of prompts by interviewers to clarify the meaning of particular terms on the survey.

4. Test of Procedures or Methods to be Undertaken

The NYTD Youth Outcome Survey was developed in consultation with practitioners, youth, and researchers in the field and was part of the proposed rule issued at 71 FR 40346.  A pilot test was conducted in August 2001 which served as a field test of the draft data elements, definitions, and procedures. It provided valuable information for assessment of the data collection burden on the States. In each of the seven pilot States, caseworkers collected data about several older youth, identified any unclear definitions, and described any difficulties encountered while collecting data. Each pilot State also was asked to report the amount of effort required to collect the information. We used these responses to assess the burden for workers, and to learn if the capacity to report data varied significantly across agencies or States.

Based on this input, we proposed a survey in the NPRM that we believed was useful to the States and balanced the burden placed on the youth with the statutory mandates for data collection. In response to the NPRM, we received very few comments from State child welfare agencies that would indicate concern that the survey will be difficult for the youth to complete. Furthermore, related studies of youth aging out of foster care, including the Multi-Site Evaluation of Foster Youth Programs, Midwest Evaluation of Adult Functioning and the Northwest Foster Care Alumni Study, conducted much more extensive surveys and typically used more personal and sensitive questions while maintaining high response rates.  On the basis of these studies and the public’s input on our rule-making, we expect that the survey as presented in the NYTD regulation will be easily understood and its content and level of burden will not discourage participation.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

John Gaudiosi (Mathematical Statistician, ACF/Children’s Bureau, 202-205-8625) was consulted on the statistical aspects of this information collection request. John Hargrove (Data Team Leader, ACF/Children’s Bureau, 202-205-8634) is responsible for primary analysis of the data associated with this information collection request. He is assisted by a team of data analysts.

References

Australian Institute of Family Studies, 2007. "Growing Up in Australia: The Longitudinal Study of Australian Children." LASC Discussion Paper no. 5, ISSN 1447-1566 (online).

Berg, N. (2002) Non-Response Bias. WWW: UTD/_nberg/Berg_ARTICLES/BergNon-ResponseBiasMay2002.pdf.

Courtney, M.E., Dworsky, A., Cusick, G.R., Keller, T., Havlicek, J., Perez, A., Terao, S. & Bost, N. (2007). Midwest evaluation of the adult functioning of former foster youth: Outcomes at age 21. Chicago, IL: Chapin Hall Center for Children at the University of Chicago.

Courtney, M.E., Dworsky, A., Cusick, G.R., Keller, T., Havlicek, J., Perez, A., Terao, S. & Bost, N. (2005). Midwest evaluation of the adult functioning of former foster youth: Outcomes at age 19. Chapin Hall Working Paper. Chicago, IL: Chapin Hall Center for Children at the University of Chicago.

Fowler, F. J. (1986). Survey Research Methods. Beverly Hills: Sage Publications

Holt, D. & Elliot, D. (1991) Methods of Weighting for Unit Non-Response. The Statistician (1991) 40, pp. 333-342.

Statistics Canada (2006). National Longitudinal Survey of Children and Youth, Cycle 6 – User Guide. Ottowa, Canada.

Turner, C.F., & Martin, E. (eds). (1984). Surveying Subjective Phenomena. New York: Basic Books.

U.S. Department of Health and Human Services, Administration for Children and Families. (2007). Multi-site evaluation of foster youth programs: Evaluation of the Life Skills Training Program. Washington, D.C.

14