EGYO Supporting Statement updated version final (2016-03-07)

Evaluation of the Young Offenders Grants

ICR Reference Number 201504-1205-006

March 2016

SUPPORTING STATEMENT PART A

Evaluation of the Young Offenders Grants

OMB Control No. 1205-0NEW

The U.S. Department of Labor’s (DOL) Employment and Training Administration (ETA) is undertaking the Evaluation of Grants Serving Young Offenders. The overall aim of the evaluation is to determine whether programs for young offenders improve youth educational and employment outcomes and reduce recidivism. ETA has contracted with Mathematica Policy Research and its subcontractor, Social Policy Research Associates, to conduct this random assignment evaluation. With this package, we request clearance for two data collection efforts to be conducted as part of the evaluation:

1. A request for consent to participate in the study (presented in Appendix A)

2. Baseline Information Form (BIF) and Contact Information Form (CIF); both presented in Appendix B)

We received clearance to conduct phone calls and site visits to select the sites for this study in May 2014 (OMB control number 1205-0436). An addendum to this package, to be submitted at a later date, will request clearance for the follow-up data collection of study participants and process study protocols. The full package for the study is being submitted in two parts because the study schedule requires random assignment to begin before the remaining instruments are developed and tested.

A. Justification

1. Circumstances Necessitating Collection of Information

Recent decades have seen a substantial increase in the number of youth arrests and a rising rate of incarceration, despite an overall decrease in juvenile crime (Brown et al. 2012). In 2008 alone, more than 2 million youth under the age of 18 were arrested (Puzzanchera 2009). Each year, of the roughly 700,000 individuals who are released from incarceration, about 200,000 are youth under age 24 (Mears and Travis 2004).

Extant research suggests that specific supports and interventions, such as enrollment in schooling or job training programs as well as access to housing, and the availability of adult mentors, mental health services, and supportive social networks, reduce the likelihood of youth recidivism and increase the chances for success among youth (Beale-Spencer and Jones-Walker 2004). It is in this context that DOL awarded funds to implement projects aimed at young adult offenders, juvenile offenders, and students in high-risk high schools, including:

• Face Forward Grants. In June 2013, ETA awarded $26 million in grant funds to 28 organizations to serve juvenile offenders ages 16 to 24. “Face Forward” conveys the idea of youth leaving behind their past transgressions and looking toward a promising future. The grantees will collaborate with nonprofit legal services providers to assist in expunging the court records of juvenile offenders and/or provide diversion programs, as designated by the juvenile justice system. The grantees have a 39-month period of performance, including a 6-month planning period. Additional grants worth $44 million will be awarded by August 2014.

• High Poverty High Crime Grants. In June 2013, ETA awarded $20 million in grants to four intermediary organizations to operate multisite projects to serve juvenile offenders and in-school youth at risk of involvement in the juvenile justice system. Youth age 14 and older, who live in high-poverty, high-crime communities are eligible. The grantees have a 39-month period of performance, including a 6-month planning period.

Although each of the youth offender grant programs is unique, important similarities will drive the evaluation. Both programs are holistic program models that offer a full complement of services including educational programs, vocational and employment programs, and mentoring. In addition, the programs target the same youth outcomes—educational attainment, improved labor market outcomes, and reduced recidivism.

These DOL-funded youth offender programs were authorized in the Workforce Investment Act (WIA). Section 171(c)(2)(A) of WIA calls for the Secretary to “carry out research projects that will contribute to the solution of employment and training problems in the United States.” This information collection is consistent with that legislative charge.

Understanding the effectiveness of the DOL-funded youth offender programs requires a rigorous evaluation that can address potential biases resulting from fundamental differences between program participants and nonparticipants. ETA has contracted with Mathematica Policy Research and its subcontractor, Social Policy Research Associates, to conduct (1) a random assignment evaluation to measure the impact of the youth offender programs and (2) a process study to understand program implementation and help interpret impact study results. Random assignment of individuals to treatment and control groups has been broadly accepted as the gold standard for providing reliable impact estimates.

This evaluation of the Young Offenders Grants will be carried out under the authority of the Public Law No. 105-220, Sec 185(d) (The Workforce Investment Act) broadly addresses reports, recordkeeping, and investigations across programs authorized under Title I of the Act. The provisions of section 185:

• require the Secretary to ensure that all elements of the information required for reports be defined and reported uniformly [section 185(d)(2)];

• direct each state, local board, and recipient (other than a sub-recipient, sub-grantee, or contractor of a recipient) to prescribe and maintain comparable management information systems, in accordance with the guidelines that shall be prescribed by the Secretary designed to facilitate the uniform compilation, cross tabulation, and analysis of programmatic, participant and financial data, on statewide, local area, and other appropriate bases, necessary for reporting, monitoring, and evaluating purposes, including data necessary to comply with section 188 [section 185(c)(2)];

• require that recipients of funds under Title I shall maintain such records and submit such reports in such form and containing such information as the Secretary may require regarding the performance of programs and activities carried out under Title I [section 185(a)(2)];

• require that recipients of funds under Title I shall maintain standardized records for all individual participants and provide to the Secretary a sufficient number of such records to provide for an adequate analysis of the records [section 185(a)(3)];

• specify that the reports shall include information about programs and activities carried out under Title I pertaining to:

- relevant demographic characteristics (including race, ethnicity, sex, and age) and other related information regarding participants;

- programs and activities in which participants are enrolled, and the length of time that participants are engaged in such programs and activities;

- outcomes of the programs and activities for participants, including the occupations of participants and placement for participants in nontraditional employment;

- specified costs of the programs and activities; and

- information necessary to prepare reports to comply with section 188 and 29 CFR Part 37.37 [(a-b), (d-e)]; and,

• require that all elements of the information required for the reports described in section 185(d)(1)(A-E) above are defined and uniformly reported.

Additionally, Public Law No. 105-220, Sec 189(d) (The Workforce Investment Act) requires the Secretary to prepare and submit to Congress an annual report regarding the programs and activities carried out under Title I. The report must include:

• a summary of the achievements, failures and problems of the programs and activities in meeting the objectives of Title I;

• a summary of major findings from research, evaluations, pilot projects, and experiments conducted under Title I in the fiscal year prior to the submission of the report;

• recommendations for modifications in the programs and activities based on analysis of such findings; and

• such other recommendations for legislative or administrative action as the Secretary determines to be appropriate.

In Figure A.1, we present a preliminary logic model describing how we envision that the youth offender programs funded by DOL affect youth; we outline some potential activities, outputs, and outcomes that we may be interested in measuring.

Figure A.1. Logic Model for Grants Serving Young Offenders

a. Overview of the Data Collection

Understanding the effectiveness of youth offender programs requires data collection from multiple sources. We propose to collect a rich set of baseline, service, and outcome data on treatment and control group members. The baseline data covered by this clearance will enable the team to describe the characteristics of study participants at the time they are randomly assigned to the treatment or control group, ensure that random assignment was conducted properly, create subgroups for the analysis, provide contact information to locate individuals for follow-up surveys, and improve the precision of the impact estimates. The documentation of the services the participants report receiving after random assignment and key outcomes of interest to the ETA (certificate attainment, employment attainment and retention, and recidivism) will be part of a future clearance package.

This request for clearance is limited to the baseline information that must be collected at the outset of the study for people who are randomly assigned. Baseline information includes characteristics of study participants collected in the BIF and detailed contact information collected in the CIF.

b. Baseline Data Collection

Prior to random assignment, baseline information will be collected for all youth who consent to be part of the study using a web-based Participant Tracking System (PTS). Baseline data elements to be collected include:

• Identifying Information. This includes complete name, address, telephone number, email, birth date, gender, and social security number (SSN) —enough information to ensure that each individual is randomly assigned only once. Identifiers are also necessary for tracking and locating sample members for follow-up surveys and for ensuring that we can obtain and accurately match administrative records on sample members. However, the contractors will inform participants that the disclosure of the SSN is voluntary and not required for participation in the evaluation.

• Demographic and Socioeconomic Characteristics and Employment History. Baseline data in these areas are required to ensure that the random assignment process was conducted properly (by confirming that the research groups have similar characteristics at baseline) and to monitor random assignment. We will also use the information to describe the study sample and to document differences in the populations served across the study sites. This information will allow us to conduct subsequent analyses of subgroups.

• Barriers to Work. The baseline data collection effort will also ask questions about specific barriers that study participants may face (children, health, poverty, unstable housing, and foster care involvement). We will use this information (1) to describe the sample and understand issues that could affect their ability to participate in training and work, and (2) to conduct subsequent subgroup analyses (for example, to determine whether the program was effective for those with specific barriers).

• Prior Criminal Justice Involvement. The baseline data collection effort will also ask questions about prior involvement with the criminal justice system. Since recidivism is a key outcome of interest for this evaluation, it is important to have good baseline measures of prior criminal justice involvement. The baseline measures will allow us to determine if the programs are effective for populations of interest, such as those with previous spells of juvenile detention, and will increase the precision of recidivism impact estimates.

• Counselor Predictions. We will also ask counselors to predict, in a designated section of the BIF, the primary services that sample members would receive if they enrolled. These data will be collected prior to random assignment, and thus, will be available for the full research sample. If these predictions are relatively accurate for the treatment group, they can be used to estimate the effectiveness of the receipt of specific program services by estimating impacts using the sample of those predicted to receive such services.

• Locating Information. Accurate locating information is crucial to achieving high survey response rates. As mentioned above, the BIF will capture each applicant’s landline and cellular telephone numbers and email address. The form will also capture social network use and alternative contact information for up to three relatives or friends who might know how to contact the sample member.

A web-based PTS will execute the random assignment procedures and compile baseline data on study sample members. This PTS will assure that participant data will be in a consistent format across sites. The PTS will also perform random assignment.

2. How, by Whom, and for What Purpose the Information Is to Be Used

Clearance is currently being requested for data collection that will be used to perform and monitor random assignment. Each form is described below, along with how, by whom, and for what purpose the information collected will be used. A subsequent addendum to this package will include a request for clearance for additional data collection instruments including follow-up data collection on sample members and implementation study protocols.

1. Consent to Participate in the Study

This form will be administered to all eligible youth participants in the selected sites by program staff at the point the youth would normally enroll in the program. The staff will ask the youth to read the form and answer any questions that the youth has prior to signing the form. The consent form, which will be returned to the evaluator, ensures that the youth has been fully informed about the study, including random assignment, all data collection, and the privacy of the data. If youth are under 18, the consent form will need to be signed by a parent or guardian. The form is presented in Appendix A.

2. The BIF and CIF

The BIF will collect basic demographic and socioeconomic characteristics on all consenting youth prior to random assignment. The CIF, an addendum to the BIF, will collect the name, address, phone number, and email address of up to three individuals who are close friends or relatives of the study participant and, most likely, will have knowledge of his or her whereabouts at the time of follow-up data collection. Both forms are presented in Appendix B.

Baseline data and contact information are needed for multiple purposes including: (1) conducting random assignment; (2) monitoring random assignment; (3) locating participants for follow-up data collection; (4) defining subgroups for impact estimates; (5) increasing the precision of impact estimates; (6) adjusting for nonresponse; and (7) estimating impacts on service recipients.

Both the BIF and the CIF will be completed by all youth who have been found eligible for enrollment and have given their signed consent (or have parental consent) to participate in the study. Program staff will work with youth to complete the forms. The information collected on these forms will be entered into a PTS, either by program staff or by evaluation staff.

3. Use of Technology to Reduce Burden

Advanced technology will be used in the data collection efforts to reduce burden on program participants and on staff at participating agencies. All program staff involved in intake will have access to the study-specific PTS. The PTS is a web-based system that can be accessed from the staff’s computer with a username and a password. It will be used to conduct random assignment and allows program staff to search for whether a youth is already participating in the study. To minimize data entry burden on the program staff, only a limited set of data items will be entered into the PTS. Hard copies of the forms will be sent to the evaluator. The evaluator will scan the forms to create electronic databases.

4. Efforts to Avoid Duplication of Effort

To minimize duplicate data collection, the BIF has been reduced to items necessary to the evaluation. Only a limited amount of descriptive information is expected to be available from the data collected electronically by programs as part of their normal intake procedures. These existing data likely do not contain all the baseline characteristics of youth necessary nor will they be consistent across sites. If discussions with sites lead to a finding that all sites collect an item on the BIF in a consistent manner, and this information is not necessary to conduct random assignment, the item can be dropped from the BIF.

The detailed contact information for the participant collected on the CIF is not available from any other source.

5. Methods of Minimizing Burden on Small Entities

The data collection effort does not involve small businesses or other small entities.

6. Consequences of Not Collecting Data

The BIF and CIF are necessary to conduct a rigorous evaluation of DOL-funded youth offender programs. Without collecting baseline information on study participants, the study’s ability to implement random assignment correctly and ensure that random assignment was conducted appropriately would be severely limited. The lack of baseline information would also limit the ability to estimate subgroup impacts and conduct nonresponse analysis. Collecting detailed contact information for study participants allows evaluators to track sample members; without this information, follow-up data collection would be limited.

7. Special Circumstances

No special circumstances are involved with the collection of information.

8. Federal Register Announcement and Consultation

a. Federal Register announcement

A 60-day notice to solicit public comments was published in the Federal Register, 79 FR 78109 on December 29, 2014. No comments were received.

b. Consultations outside the Agency

We have not consulted any experts who are not directly involved in the study regarding the subject of this clearance. We expect to consult with additional experts for other aspects of the evaluation design and impact evaluation.

9. Payment or Gift to Respondents

There are no payments to respondents. Tasks and activities conducted by program staff are expected to be carried out in the course of their employment, and no additional compensation will be provided outside of their normal pay. Sample members will not be compensated for completing study enrollment forms.

10. Privacy of the Data

Prior to random assignment, program participants will be given information about keeping information private to the extent permitted by law in the consent form that they will be asked to read and sign before being randomly assigned to a research group (see Appendix A). The information will introduce the evaluators, explain random assignment and the research groups, explain that the study participants will be asked to participate in voluntary telephone interviews, and inform participants that administrative records will be released to the research team. Participants will be told that all information provided will be kept private and used for research purposes only. Further, they will be assured that they will not be identified by name or in any way that could identify them in reports or communications with DOL.

11. Additional Justification for Sensitive Questions

The BIF will collect background information on youth who have consented to participate in this evaluation. Information on date of birth, address, and telephone numbers is needed to identify and contact sample members. The BIF also collects information on characteristics of sample members, such as sex, race/ethnicity, and education level, which is used to ensure that random assignment was conducted correctly and/or to enhance the impact estimates. This type of information is routinely collected as part of enrollment in most programs and is, therefore, not considered sensitive.

The BIF does include questions that some respondents might find sensitive. These questions ask about delinquent activities, including arrests and juvenile detentions, and physical and mental health. Collection of this information, though sensitive in nature, is critical for the evaluation and cannot be obtained through other sources. The extent of prior involvement with the criminal justice system will be an important characteristic for describing our sample members and will serve as a key subgroup for our impact analysis. Before starting random assignment, we will pretest the form to determine if any modifications need to be made. We have included similar questions in past studies without any evidence of significant harm.

As described earlier, all sample members will be provided with assurances of confidentiality prior to random assignment and the completion of study enrollment forms. Not all data items need to be completed. All data will be held in the strictest confidence and reported in aggregate, summary format, eliminating the possibility of individual identification.

12. Estimates of Hours Burden

In Table A.1, we present burden hour estimates for administering the study consent, BIF, and CIF. We base the calculations using average response times from a pilot test of data collection instruments from DOL’s WIA Gold Standard Evaluation, a random assignment evaluation that is being conducted in 28 sites aimed to measure the effectiveness of WIA’s Adult and Dislocated Worker Programs providing employment and training for low-income disadvantage job seekers as well as workers laid off from their jobs. The evaluation used similar forms to those for the current evaluation (OMB Control Number: 1205-0482), http://www.mathematica-mpr.com/our-publications-and-findings/projects/wia-gold-standard-evaluation).

Table A.1. Burden Estimates for Youth and Staff

Respondent	Number of Respondents	Frequency	Total Annual Responses	Time Per Response	Total Annual Burden (Hours)	Hourly Rate*	Monetized Value of Respondent Time
Study Consent, BIF, and CIF
Youth	5,000	1	5,000	13 min	1083	7.25	$7,852
Staffa	40	125	5,000	13 min	1083	21.79	$23,599
Unduplicated Totals	5,040	Various	10,000	13 min	2166		$31,451

^a Staff will spend approximately 13 minutes data entering the BIF and CIF into the study’s random assignment system.

* Youth hourly rate is the effective minimum wage of $7.25 as of July 2009. Available at [http://www.dol.gov/whd/minimumwage.htm]. Staff hourly rate is $21.79 based on the average wage for the “community and social service occupations” (21-0000). U.S. Department of Labor, Bureau of Labor Statistics: “National Industry-Specific Occupational Employment and Wage Estimates,” May 2014. Available at [http://www.bls.gov/oes/current/oes210000.htm].

In total, the anticipated estimate of burden on youth and staff is about 2,166 hours with a total monetized value of $31,451.

13. Estimate of Total Annual Cost Burden to Respondents or Record keepers

There are no direct costs to respondents, and there will be no start-up or ongoing financial costs incurred by respondents. The cost to respondents involves solely the time involved for the interviews and completing the baseline forms.

14. Estimates of Annualized Cost to the Federal Government

The total annualized cost to the federal government is estimated to be $177,333 over three years of baseline data collection design. Costs result from the following two categories:

• The estimated cost to the federal government for the contractor to carry out the background data collection are estimated to be $532,000 which includes the instrument development, development of the random assignment system, training for program staff, and ongoing maintenance over the course of baseline data collection. Annualized, this comes to $177,333 over three years. While these are projected costs, the actual costs to the federal government for the baseline data collection will depend on the final evaluation design including the number of sites, the number of individuals enrolled in the study, and the specific random assignment procedures that are implemented in each site.

• The annual cost borne by the DOL for federal technical staff to oversee the contract is estimated to be $23,142. The annual level of effort expected to perform these duties will require 312 hours for one Washington, D.C., based federal GS-13/7 employee earning $52.98 per hour. (See Office of Personnel Management 2016 Hourly Salary Table, available at http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2016/DCB_h.pdf). To account for fringe benefits and other overhead costs, the agency has applied a multiplication factor of 40 percent (312 hours x $52.98 x 1.4 = $23,142).

TOTAL ANNUALIZED FEDERAL COST $211,038.28. $177,333 + $23,142 = $200,475.

15. Reasons for Program Changes or Adjustments

This is a new information collection.

16. Tabulation, Publication Plans, and Time Schedules

This data collection will contribute to the Impact Findings Report.

17. Approval Not to Display the Expiration Date for OMB Approval

The expiration date for OMB approval will be displayed.

18. Exception to the Certification Statement

No exceptions to the certification statement are requested or required.

B. Collection of Information Employing Statistical Methods

This information collection request does not contain statistical methods.

REFERENCES

Beale-Spencer, M., and C. Jones-Walker. “Interventions and Services Offered to Former Juvenile Justice Offenders Reentering Their Communities: An Analysis of Program Effectiveness.” Youth Violence and Juvenile Justice, vol. 2, no. 1, 2004, pp. 88–97.

Brown, D., E. DeJesus, and V. Schiraldi. “Barriers and Promising Approaches to Workforce and Youth Development for Young Offenders: Toolkit.” Baltimore, MD: Annie E. Casey Foundation, 2012.

Mears, D.P., and J. Travis. “Youth Development and Reentry.” Youth Violence and Juvenile Justice, vol. 2, no. 1, 2004, pp. 3–20.

Puzzanchera, C. “Juvenile Arrests 2008.” Juvenile Justice Bulletin, 2009. Available at www.ojp.usdoj.gov.

U.S. Department of Labor, Bureau of Labor Statistics: “National Industry-Specific Occupational Employment and Wage Estimates,” May 2014. Available on the department’s website.

Page 10 of 11