NYTD ICR Supporting Statement-010713 finalHVchangesaccepted

THE SUPPORTING STATEMENT

Specific Instructions

Please do not remove or alter the headings below

A. Justification

1. Circumstances Making the Collection of Information Necessary

Public Law 106-169, enacted December 14, 1999, established the John H. Chafee Foster Care Independence Program (CFCIP) at section 477 of the Social Security Act (the Act). The law provides States with flexible funding to carry out programs that assist young people in making the transition from foster care to self-sufficiency. The law requires the Administration for Children and Families (ACF) to develop a data collection system to track the independent living services States provide to youth and develop outcome measures that may be used to assess States’ performance in operating their independent living programs. We are implementing this requirement by creating a collection of information entitled the National Youth in Transition Database (NYTD). The regulations for the NYTD are found at 45 CFR 1356.80. There are two information collection instruments associated with this request: the Data File and the Youth Outcome Survey. Pursuant to the regulation, States began collecting NYTD information on October 1, 2010 and submit data to ACF on a semiannual basis.

2. Purpose and Use of the Information Collection

States report information about certain youth both in and out of foster care and the services that are traditionally offered to young adults in preparation for emancipation. States use the information to improve their independent living programs to assist youth transition to independence. We use this information to track independent living services, to assess the collective outcomes of youth, and to evaluate State performance with regard to those outcomes consistent with the law’s mandate.

3. Use of Improved Information Technology and Burden Reduction

States submit NYTD data to ACF in an electronic format. We decided to use Extensible Markup Language (XML) for file transmission consistent with the E-Government Act of 2002 (Public Law 107-347) based, in part, on public responses we received during the NYTD rule-making process. ACF released its final technical specifications for the transmission of NYTD data files in June 2010 as part of NYTD Technical Bulletin #1.

4. Efforts to Identify Duplication and Use of Similar Information

Congress specifically mandated that we collect data on independent living services and youth outcomes. We have analyzed the sources and reporting instruments already in use in the collection of independent living services information and have concluded that the type of data in the NYTD is not collected elsewhere. Specifically, we examined both Federal and non-Federal data sources such as:

• Federal data collection systems such as the Adoption and Foster Care Analysis and Reporting System (AFCARS), and the Runaway and Homeless Youth Management Information System (RHYMIS);

• State, county, and local governments with integrated and/or complementary data systems such as Statewide Automated Child Welfare Information System (SACWIS); and

• Data collection efforts of non-governmental organizations such as Casey Family Programs, United Way, Lutheran Social Services, Catholic Social Services, Child Welfare League of America, and Public/Private Ventures.

AFCARS collects data on youth that are in foster care or that were adopted under the auspices of State child welfare agencies. Many youth that will be reported by States to the NYTD are also going to be reported to AFCARS. However, youth receiving independent living services may not be in foster care. Youth in the follow-up population will generally not be in foster care at ages 19 and 21. Finally, there are separate and different authorizing statutes and penalty structures for NYTD and AFCARS that do not lend themselves to combining the databases.

5. Impact on Small Businesses or Other Small Entities

This information collection is required of State agencies only and does not impact small businesses or other small entities.

6. Consequences of Collecting the Information Less Frequently

This data collection is mandated by law, therefore we would be out of compliance with the statutory requirements if we did not collect independent living service and youth outcome information. In the regulation, we require that States submit NYTD data to ACF every six months, which is the same reporting frequency as AFCARS. We believe that any less frequent reporting may increase the risk of States reporting inaccurate or missing data. Further, we chose a semi-annual reporting period to preserve our ability to analyze NYTD data along with AFCARS data for the same youth. The six-month report period for AFCARS is integral to a number of ACF priorities and legislative requirements. 

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances required in the collection of this information in a manner other than that required by OMB.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In response to the Proposed Information Collection Activity Comment Request, published in the Federal Register on April 18, 2012 (77 FR 23266), we received one public comment. Among the changes proposed by the commenter, one recommendation included changing the definition of the NYTD baseline reporting population. Because this definition was established in regulation as part of our rule-making, this recommendation is beyond the scope of consideration for this information collection activity comment request.

The commenter also disagreed with our burden estimates on States for preparing the NYTD Data File explaining that necessary administrative changes such as SACWIS enhancements and related training for caseworkers as well as outreach to contracted service providers in order to collect information on services they provide to the NYTD served population will require time that is not accounted for in our burden estimates. We are not changing our burden estimates related to States collecting and reporting information on youth independent living services as part of the NYTD Data File as suggested by the commenter. It is our expectation that States will have implemented appropriate child welfare information system enhancements by now to capture adequately the independent living services paid for or provided to youth by the State. We also account for system-related costs and training separately from the annualized burden estimates in Item 13 of this supporting statement.

The commenter also disagreed with our burden estimates related to the NYTD Youth Outcome Survey explaining that the estimate (0.50 burden hours per response) does not take into account the amount of time necessary to locate and engage youth to participate in the survey. As explained in Item 12, we did factor in burden hours associated with a State tracking youth who are transitioning out of foster care, applying these burden hours to the Data File information collection instrument and not the Youth Outcome Survey instrument. We believe the burden estimates in this ICR are still accurate based on previous studies involving locating hard to reach youth populations (Meyers, 2002) and estimates from ACF’s own Multisite Evaluation of Foster Youth Programs (Pergamit, 2012). Also, we continue to receive feedback from States, including the lone commenter, that the burden hour estimate in the current ICR for the administration of the survey (0.50 hours for youth to complete the survey) is accurate. As we note later in Item 12, this estimate (0.50 hours) is a slight decrease from the estimate in the prior approved collection (0.75 hours).

Finally, the commenter recommended utilizing email for survey invitation and web-based survey entry to minimize burden on both youth respondents and State staff responsible for administering the survey. We agree that web-based technologies have great potential to assist States in locating and engaging youth and young adults in the NYTD Youth Outcome Survey in an efficient and cost-effective way. While we have not required a specific method States must use to collect the survey data or track youth that leave foster care, we continue to provide technical assistance and other guidance to States on appropriate and allowable methods for locating youth and administering the survey (Practical Strategies for Planning and Conducting the National Youth in Transition Database (NYTD) Youth Outcome Survey).

9. Explanation of Any Payment or Gift to Respondents

No payments, other than Federal financial participation, will be made to States for the maintenance and development of an information system.

10. Assurance of Confidentiality Provided to Respondents

The final rule requires States to use an encrypted personal identification number so the identity of the individual youth remains confidential to anyone other than the State.

11. Justification for Sensitive Questions

Congress specifically required in law that States collect outcome information on issues of a sensitive nature including rates of non-marital childbirth, incarceration and high risk behaviors. Therefore, we pose questions in the youth survey that ask about these areas of a youth’s life. The survey is voluntary and we expect States to obtain youth or parental consent, as appropriate.

12. Estimates of Annualized Burden Hours and Costs

This information collection will be comprised of:

1. Data File: The State’s submission to ACF of two semi-annual data files that contain information on all data elements regarding youth services, demographics, characteristics and outcomes; and,

2. Youth Outcome Survey. A survey composed of up to 22 questions on youth outcomes (that correspond with 22 data elements in the first instrument) to be completed by youth in the baseline and/or follow-up populations.

Annual Burden Hour Estimates for the NYTD, FY 2013 – FY 2015

The following are burden hour estimates for FY2013, FY2014, and FY2015. Please note that the average annualized burden estimates (covering all three years of this information collection request) are detailed in the table labeled “Annualized burden estimates for the NYTD for the 3-year period corresponding to this ICR, FY 2013 – FY 2015” on page 7 of this document.

1. FY 2013: With regard to the Data File instrument, we estimate that there will be approximately 1,924 youth who receive services annually per State. Each State will expend on average 30 minutes (0.5 hours) to collect the services, demographics and characteristics information from those youth resulting in an hour burden of 962 per State each report period. In FY 2013, States will collect outcomes information on the follow-up population of 19-year-olds. The age 19 follow-up population is a subset of the NYTD baseline population as it includes only youth who participated in the survey at age 17. Pursuant to the NYTD regulation, some States may opt to collect outcomes information on a sample of 19-year-olds instead of the complete follow-up population. For these reasons, our estimate for the age 19 follow-up population (8,000 nationally or 154 per State; with States surveying about half of this number each report period during FY 2013) will be much smaller than our estimate for the original age 17 baseline population. We are estimating 45 minutes (0.75 hours) for States to explain the purpose of the survey to the youth and to record the outcomes information that will be reported in the data file. This estimate represents a small increase in the burden we estimated for the same activity in 2009 as we have learned that States often had to follow-up with a young person who provided conflicting or missing responses to certain survey questions. This results in a burden estimate of 58 hours per State per report period (“per response”). In addition, we are estimating tracking burden during this year as we expect most 19-year-olds to have exited foster care. Based on previous studies involving locating hard to reach youth populations (Meyers, 2002) and lessons learned from the Multisite Evaluation of Foster Youth Programs (Pergamit, 2012), we are estimating States making up to 12 contacts in FY 2013 to locate and engage a 19-year-old youth in the NYTD survey. Estimating 10 minutes per contact, we anticipate 2 burden hours for tracking 19-year-old youth in FY 2013 or 154 burden hours per State per report period. That total burden hour estimate for the Data File instrument is 1,174 per response. With regard to the Youth Outcome Survey instrument, we are estimating 0.5 burden hours for each youth to listen to the State’s survey instructions and to provide responses to the survey questions, resulting in a total of 4,000 burden hours. This estimate represents a small decrease in the burden we estimated for the same activity in 2009 as we have learned youth experienced few problems in completing the survey in 30 minutes.

FY2013

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	52	2	1,174	122,096
Services Outcomes Tracking			962	0
			58*	0
			154	0
2. Youth Outcome Survey	8,000	1	0.5	4,000
TOTAL burden for both collections				126,096

*number is rounded

2. FY 2014: With regard to the Data File instrument, we have estimated the same 962 hour burden per State per report period associated with collecting and reporting youth independent living services, demographics and characteristics information. In FY 2014, States will begin surveying a new cohort of youth starting with outcomes data collection for on another baseline population of 17-year-olds in foster care. We are estimating 45 minutes (0.75 hours) for States to explain the purpose of the survey to these youth and to record the outcomes information that will be reported in the data file. This estimate represents a small increase in the burden we estimated for the same activity in 2009 as we have learned that some States had to follow-up with a young person who provided conflicting or missing responses to certain survey questions. We estimate that there will be on average 577 17-year-old youth in foster care per State in FY 2014, resulting in approximately 216 burden hours to record this information each report period (assuming that States survey half of the baseline population of 17-year-olds in each of the two semi-annual reporting periods in FY 2014). We are not estimating tracking burden during this year for the baseline population as we expect States to know the whereabouts of the 17-years-olds still in foster care. However, we are estimating tracking burden during FY 2014 for the follow-up population as we expect States to maintain contact with this population between survey rounds at ages 19 and 21. In the intervening year between survey rounds, we estimate that States will make three contacts in FY 2014 to maintain a current address and contact information for such youth in each State’s NYTD follow-up population or follow-up population sample (154 youth per State on average). Estimating 10 minutes per contact, we anticipate 0.5 burden hours for tracking 19-year-old youth in FY 2014 or 38.5 burden hours per State per report period. The burden estimate for the Data File instrument is 1,216.50 burden hours per response or 126,516 total burden hours for 52 respondents submitting 2 responses each for FY 2014. With regard to the Youth Outcome Survey instrument, we are estimating that it will take the estimated 30,000 youth nationwide approximately 30 minutes (0.5 hours) to complete the survey, resulting in a total of 15,000 burden hours. This estimate represents a small decrease in the burden we estimated for the same activity in 2009 as we have learned youth experienced few problems in completing the survey in 30 minutes.

FFY 2014

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	52	2	1,216.5	126,516
Services Outcomes Tracking			962	130,936
			216*	19,968
			38.5	0
2. Youth Outcome Survey	30,000	1	0. 5	15,000
* rounded TOTAL burden for both collections				141,516

3. FY 2015: With regard to the Data File instrument, we have estimated the same 962 hour burden per State per report period associated with collecting and reporting youth independent living services, demographics and characteristics information. In FY 2015, States will collect outcomes information on the follow-up population of 21-year-olds. We are using the same estimate we used for this same population at age 19 (8,000 nationally or 154 per State; with States surveying about half of this number each report period during FY 2015). We are estimating 45 minutes (0.75 hours) for States to explain the purpose of the survey to these youth and to record the outcomes information that will be reported in the data file. This estimate represents a small increase in the burden we estimated for the same activity in 2009 as we have learned that States often had to follow-up with a young person who provided conflicting or missing responses to certain survey questions. This results in a burden estimate of 58 hours per State per report period. In addition, we are estimating tracking burden during this year as we expect most 21-year-olds to have exited foster care. We are estimating States making up to 12 contacts in FY 2015 to locate and engage 21-year-old youth in the NYTD survey. Estimating 10 minutes per contact, we anticipate two burden hours for tracking 21-year-old youth in FY 2015 or 154 burden hours per State per report period. Similarly, we are estimating tracking burden during FY 2015 for the second cohort of baseline population youth (surveyed in FY 2014 at age 17) as we expect States to maintain contact with this population between survey rounds at ages 17 and 19. In the intervening year between survey rounds, we estimate that States will make three contacts in FY 2015 to maintain a current address and contact information for such youth (once again, using 154 youth per State as an estimate). Estimating 10 minutes per contact, we anticipate 0.5 burden hours for tracking 18-year-old youth in FY 2015 or 38.5 burden hours per State per report period. The total burden hour estimate for the Data File instrument is 1,174 per response. With regard to the Youth Outcome Survey instrument, we are estimating 0.5 burden hours for each youth to listen to the State’s survey instructions and to provide responses to the survey questions, resulting in a total of 4,000 burden hours. This estimate represents a small decrease in the burden we estimated for the same activity in 2009 as we have learned youth experienced few problems in completing the survey in 30 minutes.

FFY 2015

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	52	2	1,212.50	126,100
Services Outcomes Tracking			962
			58*
			192.5
2. Youth Outcome Survey	8,000	1	0.5	4,000
*rounded TOTAL burden for both collections				130,100

Annualized burden estimates for the NYTD for the 3-year period corresponding to this ICR, FY 2013 – 2015

Instruments (subcomponents)	Number of respondents	Number of responses per respondent	Average burden per response (hrs)	Total burden hours (hrs)
1. Data File	52	2	1,201	124,904
2. Youth Outcome Survey	15,334*	1	0.5	7,667
*rounded TOTAL annualized burden estimate for both collections				132,571

Annualized Cost Estimates for the NYTD, FY 2013 – FY 2015

Using $31.00 per hour as an approximation for State worker hourly wages, the costs associated with gathering the Youth Outcome Survey information and preparing and submitting the NYTD Data File would be $72,788 per State for FY 2013 (1,174 burden hours per response x 2 responses per year x $31.00 hourly wage) and $75,423 per State for FY 2014 and FY 2015 (1,216.50 burden hours per response x 2 responses per year x $31.00 hourly wage). We are not estimating any costs to the youth who participate in the NYTD Youth Outcome Survey during FY 2013-2015.

Annualized cost estimates for the NYTD for the 3-year period corresponding to this ICR, FY 2013 – 2015

Instruments (subcomponents)	Total cost
1. Data File	$223,634*
FY 2013	$72,788
FY 2014	$75,423
FY 2015	$75,423
2. Youth Outcome Survey	$0
FY 2013	$0
FY 2014	$0
FY 2015	$0

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

We expect that on average business process costs, travel and training, process development, information technology (IT) operational and maintenance costs will be approximately $250,000 per State per year. Because many of these costs can be matched at 50% Federal financial participation for a State that has a SACWIS, we are estimating a cost burden of $125,000 per State per year (or $62,500 for each of the two responses per year). We do not expect that reporting for NYTD will require the purchase of additional hardware or other capital equipment. For IT changes, the main costs are in software development and/or modification. We characterize these costs as minor, but we are aware of the impact the cost of implementation has on States. During the rule-making process, we received comments from States regarding anticipated start-up and annual costs that indicated our estimates are accurate. We have applied these estimates to the cost burden calculation on item 11 of the ICRAS Jr. Part II for the NYTD Data File under “reporting.” We are not estimating any other cost burden associated with the Youth Outcome Survey instrument because we are not estimating any costs to respondent youth and because costs associated with a State collecting survey data is captured as part of other IT system development costs in the Data File instrument (as shown below).

Estimates of other total annual cost burden for reporting for the 3-year period corresponding to this ICR, FY 2013 – FY 2015

Instruments (subcomponents)	Cost per response	Responses per year	Number of respondents	Total cost burden for reporting
1. Data File	$62,500*	2	52	$6,500,000
2. Youth Outcome Survey	$0	1	15,334	$0

*$250,000 estimate per year, matched at 50% Federal financial participation = $125,000 per year or $62,500 for each response per year.

We recognize that the reporting and recordkeeping burden is disproportionately higher for small States because they need to develop the same functionality as large States regardless of the number of youth reported. State costs are not affected by the number of youth in the reporting population or the number of reporting periods. We also recognize that county-administered States may face more challenges in implementation; but saw no need to change the rule in response. Each State will have different costs depending on their populations of youth and the changes required in their information systems and business procedures to meet the requirements in the regulation.

14. Annualized Cost to the Federal Government

A Federal NYTD system was developed and implemented in 2011 to receive, process and store NYTD data submitted by States. The costs to operate, upgrade, maintain and secure the Federal NYTD system in the ACF network environment is estimated at $620,000 per year.

15. Explanation for Program Changes or Adjustments

We are not proposing any material changes in the collection instruments, instructions, or the uses of the information collection currently approved by OMB.  However, as we noted in Item 12 above, there is an adjustment in our agency’s estimate related to the Data File burden (to account for tracking youth who age out of foster care) and to the Youth Outcome Survey burden (to decrease our estimate of the amount of time youth need to complete the survey and to increase our estimate of the amount of time States need to explain the survey and collect responses from youth). We also made adjustments to our estimates related to the Youth Outcome Survey burden to account for first-ever estimates on the size of the national follow-up population cohort of youth to be surveyed at ages 19 and 21. We did not account for these estimates in our prior approved collection because the required data collection schedule for NYTD is staggered and tracking and surveying youth in the second and third survey waves had not yet begun. Based on these adjustments, we are increasing the estimate of annual responses from 13,278 to 15,334 due to the increase in the size of the respondent pool and increasing our annual burden hour estimate from 75,490 to 132,571 hours. The increase in annual dollar burden associated with these adjustments $2,125,000 (from $4,375,000 to $6,500,000).

16. Plans for Tabulation and Publication and Project Time Schedule

We plan to make annual data reports available on the Children’s Bureau website. While a complete project time schedule is still under development at this time, we released the first analysis of NYTD data on September 26, 2012.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The Department is requesting that the OMB number and expiration date not be displayed for the NYTD Data File as these files will be transmitted electronically to ACF. (Please note that we do currently post the OMB number and expiration date on the web portal States use to submit the NYTD file). The Department is also requesting that the OMB number and expiration date not be displayed on the NYTD Youth Outcome Survey. While we chose to regulate the survey questions and response options in the Youth Outcome Survey, there is no specified form or format in which States must administer the survey to youth. Instead, the Department requests permission to display the OMB number and expiration date in the regulation by technical amendment.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

B. Statistical Methods (used for collection of information employing statistical methods)

1. Respondent Universe and Sampling Methods

The NYTD Data File instrument collects semi-annual information on all NYTD data elements regarding youth services, demographics, characteristics and outcomes. Fifty-two respondent States will collect this information on an ongoing basis. No statistical methods are used or required for this instrument other those used for the Youth Outcome Survey, which is a component of the Data File.

The potential respondent universe for the Youth Outcome Survey instrument consists of 17-year-olds who are in State foster care systems during a Federal fiscal year, beginning in FY 2011, with a new cohort selected every three years thereafter (see 45 CFR 1356.81(b) and 1356.82(a)(2)). Youth that are incarcerated or institutionalized in a psychiatric facility or hospital are not a part of the baseline population because they are not in foster care consistent with the definition found in 45 CFR 1355.20. According to NYTD data from FFY 2011, approximately 25,000 youth met the definition for baseline population membership in Cohort 1. We estimate about 30,000 youth will be reported in the baseline population in FFY 2014 for Cohort 2.

Depending on the number of actual baseline respondents in a State, the State may opt to sample respondents for the follow-up population after completing the baseline year of data collection. The sampling formula is regulated in 45 CFR 1356.84. The sampling universe consists of youth in a State’s baseline population who participated in outcomes data collection at age 17. Simple random sampling procedures based on random numbers generated by a computer program is the required method unless another accepted methodology is approved by ACF. NYTD Technical Bulletin #5 specifies that ACF will draw the sample for each State that opts to sample consistent with the requirements at 45 CFR 1356.84.

The survey response rate for the baseline population in FY 2011 was approximately 60%. As States continue to apply lessons learned from the first NYTD survey, we estimate that the national response rate for the follow-up population will be 70% of 19-year-olds and 60% of 21-year-olds. To comply with the NYTD data standards, States must achieve response rates of at least 60% of 19- or 21-year olds who have aged out of foster care.

2. Procedures for the Collection of Information

The NYTD Data File instrument collects semi-annual information on all NYTD data elements regarding youth services, demographics, characteristics and outcomes. No statistical methods are used or required for this instrument other than those used for the Youth Outcome Survey, which is a component of the Data File.

As stated in the response to B.1, States will conduct the Youth Outcome Survey on a three year wave basis, starting with a new universe of 17-year-olds every three years. After States establish their first baseline population cohort in FY 2011, States that choose to sample will employ simple random sampling or they may request ACF approval of another accepted sampling methodology. ACF will not accept proposals for non-probability sampling methodologies, but will consider stratified random sampling and other probability samples that generate reliable State estimates. The sampling universe will consist of the total number of youth in the baseline population that participated in the data collection at age 17.

States will administer to youth the survey located in appendix B to the regulation. States have the discretion to conduct the surveys via in-person interviewers, computer-aided devices, phone interviews or other methods as it suits their particular needs and population. There are no dedicated resources under 42 USC 677 for States to devote to this data collection effort and funds will likely come from a combination of funds that would otherwise be used for youth independent living services and other existing resources. Given these limited State resources, and our need for data primarily as an administrative database and oversight tool, we declined to prescribe a particular survey method as is commonly used in research practices. Through technical assistance, ACF will also encourage States to use methods that are likely to achieve high response rates. We anticipate that most States will use in-person interviews or a combination of in-person and computer-aided techniques given the recent findings from studies such as the Multi-Site Evaluation of Foster Youth Programs that indicated high response rates with these approaches (U.S. Department of Health and Human Services, 2007).

Through conference calls, site visits, annual meetings and written publications, ACF has provided technical assistance to States to encourage best practices in tracking youth and administering the survey regardless of the method chosen. Attached to this request package is ACF’s current guidance to States on administering the survey (See Practical Strategies for Planning and Conducting the National Youth in Transition Database (NYTD) Youth Outcome Survey). Technical assistance on sampling will be conducted primarily with ACF statisticians, while assistance with tracking youth and administering the survey will be provided through our technical assistance partners, the National Child Welfare Resource Center for Youth Development and the National Resource Center for Child Welfare Data and Technology.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Our original expected response rates devised in 2007 were modeled on RR2 response rate (American Association for Public Opinion Research, 2006) and were reflective of our analysis of information from data collection efforts on former foster youth sponsored by ACF and States. While we had anticipated a 90% response rate from our baseline population of 17-year-old youth in foster care, only about 75% of such youth participated in the NYTD survey in FY 2011. While this population is easy to locate because they are in the placement and care responsibility of a State agency, we note that some youth could not be located in time to take the NYTD baseline survey because they had run away from foster care, did not respond to an invitation to participate in the survey, or because a “gatekeeper” such as a foster parent or group home delayed the State in gaining access to the youth. In addition, a small percentage of youth declined to participate in the survey. We believe that this baseline response rate will improve in FY 2014, as States continue to improve their survey methodology based on lessons learned from FY 2011. our anticipated response rate is suitable for our purposes, which is to have some outcomes information to meet the statutory mandates at 42 USC 677 that can provide a perspective on how youth are faring as they prepare to leave foster care and assess State performance of their independent living programs.

Approaches to collecting the Youth Outcome Survey data vary as States generally have selected the most appropriate approach to meet the needs of the State and the particular characteristics of the State's population (collection of data in-person, by telephone, using computer-aided devices, etc.). While each approach to data collection has the potential for non-response bias, response bias, and measurement error, there also are standards of practice for collecting data to address potential bias. The Children's Bureau has addressed these data quality threats through technical assistance products such as national conference calls, webinars and publications like Planning for the Mode of Administration for the Youth Outcome Survey and

Surveying Youth with Special Needs or Limited English Proficiency.

Survey researchers can and do use information on differential response rates to create weights that are used to correct bias in the data due to non-responders (Holt and Elliot, 1991), (Nathan Berg, 2002). After the Children's Bureau receives the NYTD data from the States, these corrections can be done at the Federal level using standard methods. We intend to conduct non-response analysis using data available from NYTD on demographic characteristics and independent living service history and AFCARS data regarding disabling conditions, length of time in foster care, foster care placement setting history and other factors.

Measurement error also can occur due to the respondent's inability to understand certain questions. Because of the likelihood of a wide range of comprehension levels in the target population, the TA staff will provide States with advice in dealing with this issue to ensure the most accurate collection of the information from the target youth. We will continue to provide technical assistance, for example, on the use of prompts by interviewers to clarify the meaning of particular terms on the survey.

4. Test of Procedures or Methods to be Undertaken

The NYTD Youth Outcome Survey was developed in consultation with practitioners, youth, and researchers in the field and was part of the proposed rule issued at 71 FR 40346.  A pilot test was conducted in August 2001 which served as a field test of the draft data elements, definitions, and procedures. It provided valuable information for assessment of the data collection burden on the States. In each of the seven pilot States, caseworkers collected data about several older youth, identified any unclear definitions, and described any difficulties encountered while collecting data. Each pilot State also was asked to report the amount of effort required to collect the information. We used these responses to assess the burden for workers, and to learn if the capacity to report data varied significantly across agencies or States.

Based on this input, we proposed a survey in the NPRM that we believed was useful to the States and balanced the burden placed on the youth with the statutory mandates for data collection. In response to the NPRM, we received very few comments from State child welfare agencies that would indicate concern that the survey will be difficult for the youth to complete. Furthermore, related studies of youth aging out of foster care, including the Multi-Site Evaluation of Foster Youth Programs, Midwest Evaluation of Adult Functioning and the Northwest Foster Care Alumni Study, conducted much more extensive surveys and typically used more personal and sensitive questions while maintaining high response rates.  On the basis of these studies and the public’s input on our rule-making, we expect that the survey as presented in the NYTD regulation will be easily understood and its content and level of burden will not discourage participation.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

John Gaudiosi (Mathematical Statistician, ACF/Children’s Bureau, 202-205-8625) was consulted on the statistical aspects of this information collection request. Tammy White (Data Team, ACF/Children’s Bureau, 202-205-8371) is responsible for primary analysis of the data associated with this information collection request.

References

Australian Institute of Family Studies, 2007. "Growing Up in Australia: The Longitudinal Study of Australian Children." LASC Discussion Paper no. 5, ISSN 1447-1566 (online).

Berg, N. (2002) Non-Response Bias. WWW: UTD/_nberg/Berg_ARTICLES/BergNon-ResponseBiasMay2002.pdf.

Courtney, M.E., Dworsky, A., Cusick, G.R., Keller, T., Havlicek, J., Perez, A., Terao, S. & Bost, N. (2007). Midwest evaluation of the adult functioning of former foster youth: Outcomes at age 21. Chicago, IL: Chapin Hall Center for Children at the University of Chicago.

Courtney, M.E., Dworsky, A., Cusick, G.R., Keller, T., Havlicek, J., Perez, A., Terao, S. & Bost, N. (2005). Midwest evaluation of the adult functioning of former foster youth: Outcomes at age 19. Chapin Hall Working Paper. Chicago, IL: Chapin Hall Center for Children at the University of Chicago.

Fowler, F. J. (1986). Survey Research Methods. Beverly Hills: Sage Publications

Holt, D. & Elliot, D. (1991) Methods of Weighting for Unit Non-Response. The Statistician (1991) 40, pp. 333-342.

Meyers, K., Webb, A., Frantz, J., Randall, M. What does it take to retain substance-abusing adolescents in research protocols? Delineation of effort required, strategies undertaken, costs incurred, and 6-month post-treatment differences by retention difficulty. Drug and Alcohol Dependence 69 (2003) , pp. 73-85.

Pergamit, M.R. (2012). Locating and Engaging Youth after They Leave Foster Care: Experiences Fielding the Multi-Site Evaluation of Foster Youth Programs. Washington, DC: Urban Institute.

Statistics Canada (2006). National Longitudinal Survey of Children and Youth, Cycle 6 – User Guide. Ottowa, Canada.

Turner, C.F., & Martin, E. (eds). (1984). Surveying Subjective Phenomena. New York: Basic Books.

U.S. Department of Health and Human Services, Administration for Children and Families. (2007). Multi-Site Evaluation of Foster Youth Programs: Evaluation of the Life Skills Training Program. Washington, D.C.