Appendices

Supporting Statement for OMB Clearance Request: Appendices

Innovative Strategies for Increasing Self-Sufficiency (ISIS)

Contract # HHSP23320072913YC

April 11, 2011

Prepared for

Brendan Kelly

Office of Planning, Research &

Evaluation

Administration for Children & Families

U.S. Department of Health and Human

Services

Prepared by

Abt Associates Inc.

Exhibit AA-1: The Basic Career Pathways Model

Exhibit AA-2: Theory of Change for Comprehensive Career Pathways Programs

Agreement to Take Part in the Innovative Strategies for Increasing Self-Sufficiency (ISIS) Study

We invite you to take part in an important study of services that may help people improve their skills, find a job, and advance in their careers. The study is the Innovative Strategies for Increasing Self-Sufficiency (ISIS) project. The U.S. Department of Health and Human Services is funding the project. A research company called Abt Associates is conducting the study.

[Name of site] is trying innovative approaches to help individuals improve their education and employment outcomes. To learn about how well the program is working, we will compare participants who receive the innovative program services to those who receive other services. Your participation in ISIS is voluntary. You can end your participation at any time. There are no penalties for declining to participate. We hope you will agree to be in this study. It will help us learn how well the program is working and how to improve services for future participants.

What Does Participation Mean?

If you agree to be in the study, we will assign you to one of two groups. One group will be able to participate in [Program Name]. The other group will not be able to participate in this program but may be eligible for other services in the community. We will use a lottery-like procedure to determine which group you will be in. This procedure makes sure that assignments are fair. Everyone who agrees to join the study has the same chance of receiving [Program Name] services. People who are not selected for this program can re-apply after a period of 48 months. If you decide not to participate in the study, you will not be able to get these services at [name of site]. We expect to recruit 10,800 participants across several programs to be in this study.

If you agree to be in the study, ISIS and program staff will collect information from you to help understand how well the services you receive are working. We will ask you to fill out two forms today. They will take about 40 minutes to complete. They will ask about your educational background, family structure, work history, and other experiences. You may feel that some questions are personal or sensitive. The information will help us to understand what contributes to peoples’ success, and we encourage you to provide it. You may refuse to answer any question. Your answers will not affect your placement into the program or non-program group.

The ISIS team will collect additional information about you in the future, regardless of which group you are in. This information will help us understand how well the program is working.

1) We are likely to ask you to participate in one or more additional surveys sometime in the next few years. You will receive a payment for your time completing each survey. You can refuse to participate in the interviews or answer any of the questions.

2) We will collect information about your job earnings from data maintained by this program and government agencies. This information may include information about government services you have received, such as TANF or SNAP (food stamps), and school records. To do this, we need to collect your Social Security Number.

3) We are interested if programs like [name of site] benefit children. If you have children, we may collect information about them from school records and other agencies. We also may ask your permission to talk with or observe your children. Your participation in these activities is voluntary. You or your children can choose to stop participating at any time.

Potential Risks

We are committed to keeping your personal information private. However, there is a small risk of a breach of confidentiality. We will take strong precautions to make sure this does not happen. We will keep any paper that includes your name and other personal identifiers in a locked storage area and destroy the files after the study ends. We will password protect any computer files with personally identifiable information and will store them on a secure network. We will provide the Department of Health and Human Services with a copy of the data set. The data set will not be available to the public. Your personally identifiable information will not appear in the data set or any public document produced as part of the study. Your information will be used only for the purpose of the study and will be kept confidential to the extent allowed by law.

Questions about participation

If you have any questions about the study, contact [name of site contact or liaison to be added] at ISIS@abtassoc.com (email) or [contact information to be added] (phone). If you have any questions about your rights as a study participant, contact Teresa Doksum with the Abt Associates Institutional Review Board at IRB@abtassoc.com (email) or 877-520-6835 (toll-free).

Agreement to Participate

By signing this participation agreement, I confirm that I have read and understand the description of the ISIS study. I have had the opportunity to ask questions. I understand I will be put into one of two groups at random. One group will get the innovative program services. The other group will have access to other services. I understand that my participation is voluntary. I understand that I can refuse to answer any questions or stop being in the study at any time without penalty. I understand that I will be given a copy of this consent form to keep. I understand that Abt Associates will get information about me as described above. I understand this information will be used only for the purpose of the study and will be kept confidential to the extent allowed by law.

Participant:

______________________________________

Name of Participant (Printed)

______________________________________

Signature of Participant Date

Q #	Question	Source	Justification
Identifying Information
1.	What is your name?	Adapted from Census	To be used with other contact information for tracking purposes.
2.	What is your date of birth?	Adapted from Census	Respondents are asked to identify their dates of birth to verify that programs have enrolled adults and because age may be an essential factor affecting adult learning. At some point, people may become too old to effectively change learning habits, potentially limiting the effectiveness of career pathway programs.1 On the other hand, some qualities conducive to learning, such as self-discipline, increase with age. When determining program impacts, it will be important for researchers to see if career pathway programs are more effective for some age groups than others.
3.	What is your social security number?	Adapted from Census	Researchers will use Social Security Numbers to link individuals to other administrative data sets. These data will include individuals’ earnings and use of government programs.
4.	What is your sex?	Adapted from Census	There is a substantial body of literature on occupational and wage discrimination based on sex. It is possible that women will not see the same earnings outcomes as men.
Background and Family Characteristics
5.	What is your marital status?	American Communities Survey (ACS)	Family structure could play an important role in both training and employment outcomes. Those who are married could have more support at home to allow them to focus more on training activities.
6.	Are you of Hispanic, Latino, or Spanish origin?	ACS	There is a substantial literature on occupational segregation and wage discrimination on the basis of race and ethnicity, and career pathway programs may benefit certain groups more than others.2 There is evidence that shows some education and labor force attachment programs are especially effective for minorities.3 Some of the potential ISIS programs specifically focus on increasing the number of specific populations in a given industry. For example, one of Carreras en Salud’s core missions is to increase the number of Latinos in the health care profession, where they currently compose less than 2% of all LPNs and RNs.4
7.	What is your race?	Question from ACS; response categories from Supporting Healthy Marriages (SHM)
8a.	Were you born in one of the 50 US States or Washington, DC?	SHM Baseline	Being a recent immigrant to the United States presents unique challenges to individuals trying to complete career pathway programs and find employment. Recent immigrants’ comfort with cultural norms and expectations could have an impact on program outcomes. The BIF asks two questions about English language speaking and ability because individuals who have difficulty with English may have an especially hard time completing training programs offered in English and finding employment in English language environments.5 A demonstration of the I-BEST model in Washington revealed that higher-skill ESL students are more likely to benefit from the program.6 If ISIS finds similar results, it will be important for career pathway programs to target those who are most likely to succeed to maximize their impacts.
8b.	In what year did you come to live in the United States?
9a.	Do you speak a language other than English at home?	ACS
9b.	If yes, how well do you speak English?
9c.	If yes, how well do you read English?
10.	Which of the following live in your household at least half the time?	Adapted from SHM	Collecting information on respondents’ household composition has two main purposes. First, household composition could act as a barrier to education and employment. For example, parents have to arrange and pay for childcare, may have to miss work if a child is ill, and are generally under more physical and emotional stress than childless adults. The burden is especially hard for single parents. There is substantial evidence showing that it is harder for single mothers to find employment.7 By asking about family structure, researchers are better able to assess the potential difficulties some individuals face. Second, because one of the research questions focuses on the effects of career pathway programs on the well-being of children, it is necessary for the researchers to know the number and ages of children in the home. In addition to describing the household makeup, the child roster allows researchers to select a focal child in each household to track over the course of the study.
11.	How many adults age 18 or older live in your household at least half the time?	SHM
12a.	How many children under age 18 live with you at least half the time?
12b.	For how many of these children are you or your spouse the legal guardian?	Adapted from Employment, Retention, Advancement (ERA) Study
13.	Do you have any children under age 18 who do not live with you at least half the time?	National Job Training Partnership Act (JTPA) Study
Educational Background
15.	What is the highest degree of level of school you have completed?	ACS	Educational attainment is often associated with the basic academic skills necessary to find employment. Basic math and language skills are essential for most occupations, and numerous studies have shown that education is a significant predictor of employment outcomes. Evidence suggests that high school graduates who engage in job search activities do better both in terms of employment rates and on-the-job earnings than non-graduates.8 Additionally, educational attainment could be a sign of other skills associated with better educational outcomes. Individuals who complete high school may demonstrate more persistence in accomplishing tasks or may be better at socializing or working in group environments. Such “soft” skills could affect both training and employment outcomes. While there are additional questions on the SAQ that specifically address some of these soft skills, education is still a useful proxy to identify any soft skills that cannot be measured on the SAQ.
16.	Have you received a vocational or technical certificate or diploma?	New Item
17.	Have you ever attended any of the following education and training programs?	Adapted from Fragile Families
18.	What is the highest level of education that you eventually expect to complete?	Adapted from New Visions
19a.	What is the highest level of school completed by your mother (or guardian #1)?	Adapted from the Community College Survey of Student Engagement and the Student Strengths Inventory	It is possible that having family members who have had exposure to higher education could have a positive effect on individuals own educational outcomes. If someone else in their family has attended college, an adult learner may have a better idea of how to navigate the higher education system. This could mean that they are able to access financial aid more easily or that they are able to seek out other resources that will help them be successful. The BIF asks respondents about their parents’ highest level of education and if they have any siblings who have attended college to see if individuals with more exposure to higher education do better than those with less exposure.
19b.	What is the highest level of school completed by your father (or guardian #2?
20.	Do you have any brothers or sisters who have attended college?	Adapted from Opening Doors
21.	What grades did you usually get in high school?	ACT, Inc	Grade point average has consistently been shown to be a predictor of future educational performance.
Employment and Income
22a.	Are you currently working at a job for pay?	SHM Baseline	There is strong evidence showing a stable job history is an important factor in finding future employment. Being able to keep a job is likely as important as being able to find a job because job stability signals to employers that an individual will be a reliable worker. Research has found both job skills and years worked for pay are significant predictors of future employment.9 However, it is also possible that career pathway programs will be less successful for those with stronger employment histories because they already possess the skills and resources needed to find work. There is evidence that job search activities most benefit those who do not have recent work experience.10 By asking about their employment history, the ISIS evaluation can examine the extent to which employment history is a factor in determining how individuals fare in career pathway programs.
22b.	How many hours per week on average are you currently working (include all jobs)?
22c.	When did you last work?
23.	About how much do/did you usually earn per hour before taxes in your current or most recent job?	Adapted from Fragile Families	Having the resources to participate in career pathway programs is an important determinant of success. There are many costs associated with participating in career pathway programs, including tuition, the cost of course materials, and most importantly the opportunity cost associated with working fewer hours. It is likely that some individuals will be working while they are in the career pathways program. The time they devote to the program necessarily limits the amount of time they can spend earning money to support themselves through regular employment. By asking them about their income, ISIS will be able to examine if individuals with higher levels of income are more likely to succeed in career pathway programs.
24.	In the last 12 months did you or anyone in your family (your spouse or partner and any other relative who live with you) have income or benefits from any of the following sources:	Adapted from SHM Baseline
24a.	Job earnings?
24b.	Public assistance, welfare, or WIC?
24c.	Food stamps (also known as Supplemental Nutrition Assistance Program – SNAP)?
24d.	Free or reduced price lunch program?
24e.	Unemployment insurance, worker’s compensation, disability or social security benefits?
24f.	Family and friends (outside the household)?
24g.	Grants or loans for school?
25.	Last year, what was your household’s total family income?
Contact Information
26.	Please provide your contact information.	Adapted from SHM Baseline	In order to achieve high response rates in subsequent follow-up surveys, it is important for the researchers to track all ISIS participants between waves of data collection. The BIF asks respondents both for their contact information and the contact information of three people who could locate them in the future. Extensive experience of Abt’s survey division attests to the great value of multiple points of contact in increasing the chances of contacting the respondents in the future.
27.	Please provide information for three persons not living with you who can help us locate you in case you move.

Q #	Question	Source	Justification
Resource Constraints
1.	Thinking about the near future, do you expect to be going to school part-time or full-time if you are selected for this program?		A substantial body of evidence documents negative associations between resource constraints and low-income adults’ enrollment, performance, and persistence in post-secondary education and training.11 Such adults typically face greater needs to work than traditional-aged students, who are less likely to have children and more likely to have access to some support from their own parents. A number of studies have found post-secondary success to be positively related to access to financial aid and fewer work hours (particularly below the 20-hour/week threshold).12 Additional constraints arise from limited access to computers and transportation and other personal and family challenges that occur more frequently in low-income populations.13 Financial hardship can create debilitating stresses and mental health problems, as well as increasing vulnerability to more acute emergencies that can negatively affect performance and persistence in school and at work.14 Strong social supports can provide instrumental as well as emotional reinforcements to buffer the potentially destabilizing effects of financial and other stresses.15 Some potential ISIS career pathway programs targeting various populations of low-income and low-skilled adults seek to address these and associated issues in a variety of ways. Many include assistance in obtaining financial support from public benefits and financial aid, as well as personal counseling and case management to connect students to additional supportive services. Some foster supportive social environments to create additional sources of help and emotional support. Key uses of baseline measures for resources include profiling participants in each site and measuring differences in impacts on key school and work outcomes for participants with varying initial resource constraints. In general, we hypothesize that training and employment-related impacts will be less positive among participants with fewer than for those with greater initial resources. Where programs effectively address resource constraints, however, it is possible that training and employment-related impacts will be as, or possibly more, positive among initially more disadvantaged subgroups. We will be able to measure impacts on many of the same resource outcomes over time through planned follow-up surveys to see if programs did affect them. Items in this section measure a number of specific resource constraints (In addition, the BIF includes direct measures for income). We designed SAQ items 1-5 (see Appendix C for the SAQ instrument) to assess expected difficulties managing both work and school and concerns about financial support. Items 6-7 tap access to a computer and car, two resources that can be critical to success in school and finding and retaining employment. Items 8-9 are two widely-used measures of financial hardship. Items in Question 10 reproduce the short form of the widely-used Social Provisions Scale.16
2.	Do you expect to be working for pay in the next few months?
2a.	If yes, how many hours to you expect to be working in a typical week?
3.	How difficult do you expect it will be to find enough time to do well in school?
4.	While you are in school, do you expect to need financial assistance to help with:
4a.	Tuition and fees?
4b.	Living expenses?
5.	How difficult to you expect it to be to obtain enough financial support for school?
6.	Do you have a computer at home?
6a.	Does this computer have a working Internet connection?
7.	Do you own a car?
8.	In the past 12 months has there been a time when you could not pay the full amount of the rent or mortgage that you were supposed to pay?
9.	Think again over the past 12 months. Generally, at the end of each month do you end up with: more than enough money left over, some money left over, just enough to make ends meet, or not enough to make ends meet?
10.	The following statements are about help from other people. Please indicate whether you strongly disagree, disagree, agree, or strongly agree with each item.	Scale of Social Provision
10a.	There are people I can depend on to help me if I really need it.
10b.	I feel that I do not have close personal relationships with other people.
10c.	There is no one I can turn to for guidance in times of stress.
10d.	There are people who enjoy the same social activities that I do.
10e.	I do not think other people respect my skills and abilities.
10f.	If something went wrong, no one would come to my assistance.
10g.	I have close relationships that provide me with a sense of emotional security and well-being.
10h.	I have relationships where my competence and skills are recognized.
10i.	There is no one who shares my interests and concerns.
10j.	There is a trustworthy person I could turn to for advice if I were having problems.
Psycho-Social Factors
11a.	Discipline (10 items)	Each construct is part of a proprietary scale designed, used, and validated by ACT, Inc. as part of their Student Readiness Inventory or Talent Assessment tools.	The last two decades have been marked by growing interest in the psycho-social factors linked to success in post-secondary education and employment. Economists and education researchers have argued for greater attention to these factors.17 Important governmental and non-governmental initiatives followed in the wake of the 1991 DOL report of the Secretary’s Commission on Needed Skills.18 These initiatives variously have sought to map out the wide range of skills, classify occupations according to skills required, identify skills required for success in occupational training programs, and develop approaches to instilling these skills. Yet as interventions have proliferated, there has been surprisingly little effort to inventory approaches, carefully distinguish their target factors, or rigorously evaluate impacts on these factors.19 ISIS affords a valuable opportunity to build knowledge in this arena. In addition to fitting within our broader theory of change for career pathways, a number of potential ISIS sites incorporate components targeting psycho-social factors and related coping skills. Examples include job readiness training focused on work habits/communication, student success curricula, and intensive mentoring and guidance. Absent components effectively addressing these factors, we generally expect less favorable training and employment impacts for participants with initially weaker psycho-social skills. For interventions that effectively impact these skills, we expect training and employment impacts to be comparable to – and perhaps even more favorable than – those for initially more psycho-socially advantaged participants. Some evidence exists that is consistent with these hypotheses.20 Our proposed measures capitalize on perhaps the most comprehensive effort to date to conceptualize and measure psycho-social factors—a program undertaken by researchers at ACT Inc. The ACT team conducted extensive research to identify and measure psycho-social factors important in college and work settings. Their progress is summarized in a series of peer-reviewed papers, including two meta-analyses21 and reports on psychometric results on the new measures.22 The final products are embodied in two instruments: one for college settings (Student Readiness Inventory/SRI) and one for work settings (Talent Assessment).23 These instruments have been administered to thousands of students and workers, and ACT has extensive data on national norms. Based on a series of discussions with ACT staff and published psychometric evidence, the ISIS team has identified a subset of scales with the greatest relevance to adult career pathway programs (see SAQ items 11a-f). We have selected six ACT scales for inclusion: discipline, training commitment, determination/striving, self-confidence, emotional steadiness, and optimism. In each case, scales will sum a series of 10-13 items developed and validated by ACT. Illustrative items for each scale are shown in the ISIS SAQ. The full set of items is proprietary and copyrighted by ACT.24
11b.	Training Commitment (10 items)
11c.	Determination/Striving (11 items)
11d.	Self-confidence (12 items)
11e.	Steadiness (12 items)
11f.	Optimism (13 items)
Career Orientation and Knowledge
12.	When it comes to careers, some people are more certain than others that they know where they are headed and how to get there. Please indicate for each item in the list below whether you strongly disagree, disagree, agree, or strongly agree that it reflects your career situation.	Career Decisions Self-Efficacy Scale Short Form	A key assumption in the career pathways framework is that more effective guidance is needed to foster career knowledge and planning skills and equip students to maximize their access to available supports in pursuing desired training and employment.25 Low-income adults with little exposure to careers or post-secondary training are particularly likely to lack direction and skills for planning careers, obtaining financial aid, getting extra help at school when needed, and identifying appropriate employment opportunities.26 In response, career pathway programs are developing and using improved career exploration and planning instruments, providing more intensive counseling and “career navigation” supports, packaging training and supports in ways that allow students to concentrate on big picture decisions with less need to arrange myriad details, and formally instructing students in skills needed to succeed in college and work. By infusing courses on foundational academic skills (math, reading) with examples from different occupations, some programs are trying to foster awareness and interest in relevant careers while providing more engaging basic skills training. Absent such services, we expect that participants with weaker initial career orientations will not benefit from other career pathways strategies – such as occupational training and material supports – as much as those with stronger career orientations. Given strong career services, however, we might expect commensurate, or even greater, impacts on training and employment outcomes among those with weaker initial career orientations. To measure career directedness, SAQ items 12a-g elicit participants’ assessments of self-knowledge and skills in several key areas – self-assessment and planning, getting help at school, and having a sense of one’s preferred occupation and training. Items a-b are adapted from the Career Decisions Self-Efficacy Scale-Short Form,27 d-f are from the Career Exploration Survey,28 and c and g are new items.
12a.	You know how to accurately assess your abilities and challenges.	New Item
12b.	You know how to make a plan that will help achieve your goals for the next 5 years?	Career Exploration Survey
12c.	You know how to get help from staff and teachers with any issues that might arise at school?	New Item
12d.	You know the type of job that is best for you?	Career Exploration Survey
12e.	You know the type of organization you want to work for?	Career Decisions Self-Efficacy Scale Short Form
12f.	You know exactly the occupation you want to enter?	New Item
12g.	You know the kind of education and training program that is best for you?	Career Exploration Survey
Personal and Family Challenges
13.	The questions below ask you about your feelings and thoughts during the last month. The best approach is to answer each question fairly quickly, giving a reasonable estimate without trying to count up the exact number of times. In the past month, how often have you felt:	Perceived Stress Scale	Poverty and its associated financial hardships are associated with elevated levels of a variety of personal and family challenges that can interfere with success at school and work. Extensive research has documented connections between poverty, psychological stress, mental health and resulting difficulties sustaining levels of concentration and engagement needed to pursue a career and balance doing so with family responsibilities.29 Additional barriers include access to child care and transportation, substance abuse, other health conditions, family violence, and criminal background.30 Career pathways initiatives have sought to increase access to services addressing these challenges, services that traditionally have been more abundant in social service settings than in community college and other post-secondary education settings. Most of the potential programs ISIS will be testing include some form of needs assessment and case management, with varying connections to local providers and in-house services. Some programs offer workshops and training aiming to strengthen skills for coping with stress, building resilience in the face of reversals, and finding help for oneself. In addition to providing information essential to developing individual profiles on these dimensions of disadvantage across potential ISIS sites, baseline measures will support subgroup analyses to test important hypothesized moderating effects. Absent intervention provisions addressing them effectively, we expect less favorable training and employment impacts among individuals with higher levels of stress, depressive symptoms, and other personal and family challenges than among those with lower levels. With effective provisions addressing these problems, we expect more favorable impacts for more than for less initially disadvantaged groups.31 To capture key challenges, the ISIS SAQ includes three sets of items: one series measuring perceived stress, a second measuring a set of widely-studied barriers, and a third set measuring depressive symptoms. Our stress measure (SAQ Items 13a-d) is a widely used four-item version of the Perceived Stress Scale32, SAQ items 14a-f adapt the functional impairments format used to assess mental health and substance abuse problems to several other barriers33, and Item 15 is from the baseline survey used in the Employment Retention and Advancement project. Finally, Items 16a-i are a short form of the Center for Epidemiologic Studies-Depression scale (CES-D), a measure of depressive symptoms that has been widely used in comparable intervention studies.34
13a.	That you were unable to control the important things in your life?
13b.	Confident about your ability to handle your personal problems?
13c.	That things were going your way?
13d.	That difficulties were piling up so high that you could not overcome them?
14.	In the past 12 months, please note how often each of the following items interfered with important school, work, job search, or family responsibilities.	Adapted from the Composite International Diagnostic Interview (CIDI) Short Form, the Prime-MD scale, and the Employment, Retention, and Advancement (ERA) baseline study
14a.	Child care arrangements
14b.	Transportation
14c.	Alcohol or drug use
14d.	An illness or health condition
14e.	Arguments with a family member
14f.	Physical threats/violence from a family member
15.	Have you ever been arrested?	ERA baseline study
15a.	Have you ever been convicted of a crime?
15b.	Have you been convicted of a felony?
16.	Below is a list of the ways you might have felt or behaved in the last week. For each, please indicate how often you felt this way during the past week.	Center for Epidemiologic Studies-Depression Scale (CES-D)
16a.	I was bothered by things that usually don’t bother me.
16b.	I felt that I could not shake off the blues even with help from my family or friends.
16c.	I had trouble keeping my mind on what I was doing.
16d.	I felt depressed.
16e.	I felt that everything I did was an effort.
16f.	My sleep was restless.
16g.	I was happy.
16h.	I enjoyed life.
16i.	I felt sad.

References

ACT, Inc. (2007). Talent Assessment: User and Technical Guide. Iowa City, IA.

Allen, J., Robbins, S., & Sawyer, R. (2010). Can measuring psychosocial factors promote college success? Applied Measurement in Education, 23(1), 1-22.

Bound, J., Lovenheim, M. F., & Turner, S. (2010a). Increasing Time to Baccalaureate Degree in the United States. National Bureau of Economic Research, Inc, NBER Working Papers: 15892. Retrieved from http://www.nber.org/papers/w15892.pdf.

Carnevale, A., Fry, R., & Lowell, B. (2001). Understanding, speaking, reading, writing, and earnings in the immigrant labor market. American Economic Review, 91(2), 159-163.

Chaney, D., Hammond, M., Betz, N., & Multon, K. (2007). The reliability and factor structure of the Career Decision Self-efficacy Scale-SF with African Americans. Journal of Career Assessment, 15, 194-105.

Choitz, V., Soares, L., & Pleasants, R. (2010). A new national approach to career navigation for working learners. Washington, DC: Center for American Progress.

Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health & Social Behavior, 24(4), 385-396.

Colquitt, J., LePine, J., & Noe, R. (2000). Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85(5), 678-707.

Conger, R., Rueter, M., & Elder Jr, G. (1999). Couple resilience to economic pressure. Journal of Personality & Social Psychology, 76(1), 54-71.

Conley, D. (2007). Toward a more comprehensive conception of college readiness. Eugene, OR: Educational Policy Improvement Center.

Crew Jr, R., & Davis, B. (2006). Substance abuse as a barrier to employment of welfare recipients. Journal of Policy Practice, 5(4), 69-82.

Cutrona, C., & Russell, D. (1987). The provisions of social relationships and adaptation to stress. Advances in Personal Relationships, 1.

Cutrona, C., Russell, D., Brown, P., Hessling, R., Clark, L., & Garder, K. (2005). Neighborhood context, personality, and stressful life events as predictors of depression among African American women. Journal of Abnormal Psychology, 114(1), 3-15.

Danziger, S., Kalil, A., & Anderson, N. (2000). Human capital, physical health, and mental health of welfare recipients: co-occurrence and correlates. Journal of Social Issues, 56(4).

Darity Jr, W., & Mason, P. (1998). Evidence on discrimination in employment: Codes of color, codes of gender. Journal of Economic Perspectives, 12(2), 63-90.

Engstrom, C., & Tinto, V. (2008). Learning better together: the impact of learning communities on the persistence of low-income students. Opportunity Matters, 1.

Estrada, R. (2010). How to build bridge programs that fit into a career pathway. Chicago, IL: Instituto del Progreso Latino.

Fein, D., & Beecroft, E. (2006). College as a job advancement strategy: final report on the New Visions Self-Sufficiency and Lifelong Learning Project. Bethesda, MD: Abt Associates Inc.

Goldrick-Rab, S., & Sorensen, K. (2010). Unmarried parents in college. Future of Children, 20(2), 179-203.

Gottschalk, P. (2005a). Can work alter welfare recipients' beliefs? [Article]. Journal of Policy Analysis & Management, 24(3), 485-498.

Hamilton, G., Brock, T., Farrell, M., Friedlander, D., & Harknett, K. (1997). Evaluating two welfare-to-work program approaches: two-year findings on the labor force attachment and human capital development programs in three sites. New York, NY: MDRC.

Hamilton, G., Freedman, S., Gennetian, L., Michalopoulos, C., Walter, J., Adams-Ciardullo, D., . . . Ahluwalia, S. (2001). How effective are different welfare-to-work approaches? Five-year adult and child impacts for eleven programs. New York, NY: MDRC.

Heckman, J. (1998). Detecting discrimination. Journal of Economic Perspectives, 12(2), 101-116.

Heckman, J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411-482.

Hinckley, R., & Hull, D. (2007). Career-limited adults in the era of high technology and globalization. In D. Hull & R. Hinckley (Eds.), Adult Career Pathways: Providing a Second Chance in Public Education. Waco, TX: CORD.

Jenkins, D. (2006). Career pathways: aligning public resources to support individual and regional economic advancement in the knowledge economy. New York, NY: Workforce Strategy Center.

Kessler, R. (2002). Epidemiology of depression. In I. Gotlib & C. Hammen (Eds.), Handbook of Depression. New York, NY: The Guilford Press.

Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college outcomes: constructing the Student Readiness Inventory. Educational & Psychological Measurement, 65(3), 482-508.

Lee, S., & Vinokur, A. (2007). Work barriers in the context of pathways to the employment of welfare-to-work clients. American Journal of Community Psychology, 40(3/4), 301-312.

Leininger, L., & Kalil, A. (2008). Cognitive and non-cognitive predictors of success in adult education programs: evidence from experimental data with low-income welfare recipients. Journal of Policy Analysis & Management, 27(3), 521-535.

Maguire, S., Freely, J., Clymer, C., Conway, M., and & Schwartz, D. (2010). Tuning In to Local Labor Markets: Findings From the Sectoral Employment Impact Study. Philadelphia, PA: Public/Private Ventures.

Matus-Grossman, L., & Gooden, S. (2002). Students' perspectives on juggling work, family, and college. New York, NY: MDRC.

Michalopoulos, C., & Schwartz, C. (2001). What works best for whom: impacts of 20 welfare-to-work programs by subgroup. New York, NY: MDRC.

Morgenstern, J., Neighbors, C., Kuerbis, A., Riordan, A., Blanchard, K., McVeigh, K.H., Morgan, T.J., McCrady, B. (2009). Intensive Case Management Improves 24-Month Abstinence and Employment Outcomes for Substance Dependent Women Receiving Temporary Assistance to Needy Families (TANF). American Journal of Public Health, 99, 328-333.

Office of Management and Budget. (2006). Questions and answers when designing surveys for information collections. Washington, DC.

Olson, K., & Pavetti, L. (1996). Personal & family challenges to the successful transition from welfare to work. Washington, DC: The Urban Institute.

Peterson, C., Casillas, A., & Robbins, S. (2006). The Student Readiness Inventory and the big five: examining social desirability and college academic performance. Personality & Individual Differences, 41(4), 663-673.

Pleasants, R., & Clagett, M. (2010). Career pathways: background paper for a discussion of how the federal government can support their expansion. Boston, MA: Jobs for the Future.

Price, R., Choi, J., & Vinokur, A. (2002). Links in the chain of adversity following job loss: how financial strain and loss of personal control lead to depression, impaired functioning, and poor health. Journal of Occupational Health Psychology, 7(4), 302-312.

Purnell, R., & Blank, S. (2004). Support success: services that may help low-income students succeed in community college. New York, NY: MDRC.

Riger, S., Staggs, S., & Schewe, P. (2004). Intimate partner violence as an obstacle to employment among mothers affected by welfare reform. Journal of Social Issues, 60(4), 801-818.

Robbins, S., Allen, J., Casillas, A., Peterson, C., & Huy, L. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98(3), 598-616.

Robbins, S., Huy, L., Davis, D., Carlstrom, A., Lauver, K., & Langley, R. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130(2), 261-288.

Robbins, S., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: integrated meta-analytic path analyses. The Journal Of Applied Psychology, 94(5), 1163-1184.

Roderick, M., Nagaoka, J., & Coca, V. (2009). College readiness for all: the challenge for urban high schools. Future of Children, 19(1), 185-210.

Santor, D., & Coyne, J. (1997). Shortening the CES-D to improve its ability to detect cases of depression. Psychological Assessment, 9(3), 233.

Scrivener, S., Bloom, D., LeBlanc, A., Paxson, C., Rouse, C., & Sommo, C. (2008). A good start: two-year effects of a freshmen learning community program at Kingsborough Community College. New York, NY: MDRC.

Scott-Clayton, J. (2010). The Causal Effect of Federal Work-Study Assistance: Quasi-Experimental Evidence from West Virginia. Columbia University, Teachers College Working Paper. Retrieved from http://faculty.tc.columbia.edu/upload/js3676/ScottClayton_WorkStudy_08-04-2010.pdf.

Seefeldt, K., & Orzol, S. (2005). Watching the clock tick: factors associated with TANF accumulation. Social Work Research, 29(4), 215-229.

Seftor, N. S., & Turner, S. E. (2002). Back to School. [Article]. Journal of Human Resources, 37(2), 336-352.

Spitzer, R., Williams, J., Kroenke, K., Hornyak, R., & McMurray, J. (2000). Validity and utility of the PRIME-MD patient health questionnaire in assessment of 3000 obstetric-gynecologic patients: the PRIME-MD Patient Health Questionnaire Obstetrics-Gynecology Study. American Journal Of Obstetrics And Gynecology, 183(3), 759-769.

Stephens, R. (2009). Charting a path: an exploration of the statewide career pathway efforts in Arkansas, Kentucky, Oregon, Washington and Wisconsin. Seattle, WA: Seattle Jobs Initiative.

Turner, H., & Turner, R. (2005). Understanding variations in exposure to social stress. Health: An Interdisciplinary Journal for the Social Study of Health, Illness & Medicine, 9(2), 209-240.

Turner, R., & Avison, W. (2003). Status variations in stress exposure: implications for the interpretation of research on race, socioeconomic status, and gender. Journal of Health & Social Behavior, 44(4), 488-505.

Turner, R., & Lloyd, D. (1999). The stress process and the social distribution of depression. Journal of Health & Social Behavior, 40(4), 374-374.

Turner, R., Wheaton, B., & Lloyd, D. (1995). The epidemiology of social stress. American Sociological Review, 60(1), 104-125.

Vinokur, A., Price, R., & Caplan, R. (1996). Hard times and hurtful partners: how financial strain affects depression and relationship satisfaction of unemployed persons and their spouses. Journal of Personality & Social Psychology, 71(1), 166-179.

Vinokur, A., & Schul, Y. (2002). The web of coping resources and pathways to reemployment following a job loss. Journal of Occupational Health Psychology, 7(1), 68-83.

Vinokur, A., Schul, Y., Vuori, J., & Price, R. (2000). Two years after a job loss: long-term impact of the JOBS program on reemployment and mental health. Journal Of Occupational Health Psychology, 5(1), 32-47.

Washington State Board for Community and Technical Colleges. (2005). I-BEST: a program integrating adult basic education and workforce training. Olympia, WA.

Wlodkowski, R. J., Mauldin, J. E., & Campbell, S. (2002). Early Exit: Understanding Adult Attrition in Accelerated and Traditional Postsecondary Programs. Synopsis. July, Indianapolis: Lumina Foundation for Education, 2002.

1 See Leininger & Kalil (2008) for a quick summary of the literature.

2 For a debate as to the prevalence of racial discrimination see Darity & Mason (1998) and Heckman (1998).

3 The NEWWS evaluation showed that, in general, African American and Hispanic participants in human capital development and job search activities had higher earnings impacts than white participants in the same programs. Hamilton et al. (2001).

4 See Estrada & DuBois (2008).

5 See Carnevale, Fry, & Lowell (2001) found that English language has a significant impact on employment outcomes.

6 See Washington State Board for Community and Technical Colleges (2005).

7 See Crew & Davis (2006), Danziger, Kalil, & Anderson (2000), Olson & Pavetti (1996), Riger, Staggs, & Schewe (2004), and Seefeldt & Orzol (2005).

8 See Hamilton et al. (1997).

9 See Riger et al. (2004).

10 See Hamilton et al. (2001).

11 See Goldrick-Rabb & Sorenson (2010), Long (2010), Community College Survey of Student Engagement (2008), Fein & Beecroft (2006), Matus-Grossman & Gooden (2002), Wlodkowski et al. (2002).

12 See Bound et al. (2010), Scott-Clayton (2010), Long (2010), Deming & Dynarski (2009), Navarro et al. (2007), Berker & Horn (2003), Seftor & Turner (2002).

13 See Hinckley & Hull (2007), Purnell & Blank (2004), and citations under “Personal and Family Challenges” below.

14 See citations under “Personal and Family Challenges” below.

15 See Cooper (2010), Scrivener et al. (2008), Engstrom & Tinto (2008), Conger et al. (1999), Vinokur et al. (1996).

16 See Cutrona & Russell (1984, 1987).

17 For example, see Allen et al. (2010), Conley (2007), Heckman et al. (2006), and Colquitt et al. (2000).

18 U.S. Department of Labor (1991).

19 Exceptions include a random assignment test of the Michigan JOBS program (Vinokur et al 2000) and limited investigation of moderating (Leininger & Kalil 2008) and mediating (Gottchalk 2005) effects of psycho-social factors in random assignment evaluations of welfare-to-work programs.

20 Vinokur et al. (2000) found statistically significant moderating effects for baseline measures of motivation and self-mastery on employment and mental health outcomes, respectively. As expected, impacts were greater for initially less motivated/lower mastery participants for this program, which focused on building self-esteem and motivation. On the other hand, Leininger & Kalil (2008) found positive impacts on GED receipt concentrated among initially more confident welfare recipients for programs that did not target psycho-social factors.

21 See Robbins et al. (2004, 2009).

22 See Le et al. (2005), Peterson et al. (2006), Robbins et al. (2006), and ACT Inc. (2007).

23 See materials at: http://www.act.org/sri/pdf/UserGuide.pdf, http://www.act.org/workkeys/assess/talent/.

24 ISIS is developing an agreement with ACT for permission to use these instruments.

25 See for example Choitz (2010), Cooper (2010), and Conley (2007).

26 See Long (2010), Roderick et al. (2009), Purnell & Blank (2004), and Matus-Grossman & Gooden (2002).

27 See Chaney et al. (2007).

28 See Stumpf et al. (1983).

29 See Cutrona et al. (2005), Turner & Turner (2005), Turner & Avison (2003), Kessler (2002), Price et al. (2002), Vinokur & Schul (2002), Conger et al. (1999), Turner & Lloyd (1999), Vinokur et al. (1996), and Turner et al. (1995).

30 See review in Lee & Vinokur (2007), also Fein & Beecroft (2006), Grossman & Gooden (2002).

31 Some experimental evidence exists supporting these hypotheses. Impacts on earnings and welfare payments in a review of 20 random assignment tests of welfare-to-work programs generally were more favorable for participants with low initial risk of depression compared with those at high initial risk, particularly for programs with an educational focus (Michalopolous & Schwartz 2001). In contrast, a random assignment test of a psycho-educationally focused intervention for recently unemployed job seekers found both reductions in depressive symptoms and somewhat more favorable economic impacts among initially high-risk participants in the short (six months) though not longer (two years) run (Vinokur et al. 2000). There also is some experimental evidence that programs effectively treating substance abuse among low-income adults also produce more favorable employment outcomes. (Morgenstern et al. 2009)

32 Cohen et al. (1983).

33 E.g., World Health Organization Composite International Diagnostic Interview Short Form (CIDI-SF, see www3.who.int/cidi); Prime-MD, another widely used mental health screening instrument (Spitzer et al. 2000); and the Employment Retention and Advancement baseline form.

34 SAQ items from Santor & Coynes’ (1997) CES-D short form. We opted for this leading depression measure rather than another widely used general measure of psychological distress, the K6, because depression is a more clearly defined condition whose import is well established in the clinical and intervention literatures. Compared with indices developed to measure risk of major depressive episodes, the CES-D provides valid measures of sub-clinical symptoms likely to affect behavior at school, work, and home.