Frequently Asked Questions: Enclosure with Letter

E

valuation of YouthBuild
Questions & Answers for Grantees

Questions about the YouthBuild evaluation

Who will be conducting the evaluation? ETA selected a team of three organizations to conduct the evaluation: MDRC (the lead), Mathematica Policy Research, and Social Policy Research Associates (SPRA).

What will the evaluation measure? The evaluation will measure core program outcomes including educational attainment, postsecondary planning, employment, earnings, delinquency and involvement with the criminal justice system, and youth social and emotional development.

YouthBuild has been evaluated before; why is this evaluation necessary? Previous evaluations of YouthBuild have provided very useful information, but none of them were designed to determine what difference YouthBuild makes for the young people it serves compared to what would have happened if they did not participate. The new evaluation is the first one to use a random assignment design, which is considered to provide the most reliable evidence about the effectiveness of a social program.

Will the evaluation mean that programs serve fewer youth than before? Most YouthBuild programs are only able to serve a very small fraction of the youth in their communities who might benefit and who apply. Programs participating in the evaluation will assess and screen youth to identify those eligible and appropriate for YouthBuild and then use random assignment (like picking names from a hat) to select which youth will be admitted. Selected youth will form the “program group” for the study. The youth who are not selected will form the “control group” and will not be permitted to enroll in YouthBuild during the study period, though the grantees may refer them to other youth programs in the community. Ideally, the evaluation should not cause YouthBuild programs to serve fewer youth than they otherwise would.

How will grantees be selected for the evaluation and how many will be selected? Approximately 77 grantees will be selected to participate in the evaluation. Grantees will be selected randomly, but some types of grantees (for example, those that have never operated a YouthBuild program before) may be excluded from consideration. Larger programs may have a greater chance of being selected in order to ensure that the evaluation includes a sufficient number of young people.

What data will grantees be required to provide to the evaluators? The grantees will not have to do anything more than usual to stay in touch with youth who participate in their program. The evaluators will obtain most data from the DOL YouthBuild Management Information System (MIS), from government records, and from surveys of youth in the program and control groups. Grantee staff will be asked to complete two surveys and host site visits by the research team. Also, grantees will be required to collect and enter some information into the MIS for all youth entering the study, some of whom will be assigned to the control group.

Questions about the Site Selection Questionnaire

Do I have to complete this survey? Yes. As a recipient of a DOL YouthBuild grant, you agreed to participate in an evaluation of the program. This survey is a very important part of the evaluation. Therefore, the participation of all grant recipients is required.

Will my information be kept confidential? Yes. The information that we collect from you for this study will be used for research purposes only and will be kept private and confidential to the extent provided by law. Your answers will be combined with the answers of other grantee respondents..

Who can I contact for more information? For more information about the questionnaire, please call Mathematica toll free at 1-877-894-4589and ask for Shawn Marsh. For information about Mathematica, visit their website at http://www.mathematica-mpr.com. For more information about the study, please visit www.xxxxxx.com.