ISIS OMB Package B 11-01-11_clean

This document presents Part B of the Supporting Statement for the Innovative Strategies for Increasing Self-Sufficiency (ISIS) evaluation sponsored by the Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS). Abt Associates Inc. (Abt) is the prime contractor for the study. ISIS will evaluate post-secondary career pathway programs that target economically disadvantaged families and individuals.

This submission seeks clearance for three data collection instruments:

a Basic Information Form (BIF) for participants;
a Self-Administered Questionnaire (SAQ) for participants; and
an Interview Guide for interviews with program staff.

This submission seeks clearance for the baseline data collection activities. Subsequent OMB submissions will seek clearance for follow-up data collection activities.

B.1 Potential Respondent Universe and Survey Sample

The ISIS study is recruiting sites that have innovative career pathway programs in place and are able to implement random assignment tests of these programs. The sample of nine sites ultimately included in ISIS will be a purposive sample of the most promising career pathways interventions, rather than a nationally representative sample. Thus, the universe of potential respondents is low-income adults (age 18 or older) who are interested in occupational skills training and who reside in the geographical areas where ISIS sites are located. The target enrollment for the study is an average of 1,200 individuals in each site, for a total of 10,800 individuals in the study as a whole.

B.2 Procedures for Collecting the Information

B.2.1 Sample Design

The target sample size for the ISIS study is 10,800 individuals. The nine study sites will each recruit about 1,200 individuals interested in career pathways services who agree to participate in the study. Half of the sample members in each site will be assigned to the treatment group to receive the career pathways intervention and the other half will be assigned to a control group. Sample members assigned to the control group will have access to all other services normally provided in the community. We expect that control group members in community college sites will be taking at least some college courses, whereas control group members in community-based organizations are less likely to enroll in courses or other programs.

All randomly assigned individuals will be included in the participant tracking. For the 12-month follow-up survey (not included in this request for OMB clearance), interviews will be attempted with all members of the research sample (10,800 individuals). Therefore, no sampling is required for the tracking or the 12-month follow-up survey. In a later follow-up data collection (tentatively scheduled for 36 months after random assignment) we may conduct direct child assessments with one focal child per family. The sampling method to choose a focal child will be described in the future OMB package that seeks clearance for the direct child assessments.

Data to analyze the impacts of the career pathways interventions will come primarily from the administrative earnings records and the follow up surveys, which will be submitted for OMB review under a separate supporting statement. Key topics to be included in the follow-up survey are training outcomes, detailed employment outcomes not available in administrative data (such as wage rate, hours, and benefits), adult health and well-being, and health and well-being.

B.2.2 Estimation Procedures

The baseline data to be collected for the evaluation will be used to describe the study sample, to define subgroups for analysis, and to provide baseline measures of outcomes to use as covariates in impact estimates to improve precision. With properly designed and implemented random assignment, treatment-control comparisons of raw mean outcomes provide unbiased estimates of impact. Use of regression analysis to control for baseline characteristics that affect the outcome improves the precision of the estimates while preserving their unbiased character. The estimates of precision presented in the next section assume such regression adjustments, with precision gains based on those obtained in similar studies.

B.2.3 Degree of Accuracy Required

The baseline data collected will be used in the future in conjunction with follow-up survey data and administrative data to estimate impacts of career pathway interventions. The research team has estimated the minimum detectable effects for this evaluation that will be available through the impact analysis. The analysis of statistical power is presented here.

Power Calculations for Binary Outcomes

In this section, we report minimum detectable effects (MDEs), which are the smallest true effects of an intervention that researchers can be confident of detecting as statistically significant when analyzing samples of a given size. Our analysis indicates that the proposed design will have sufficient statistical power to detect impacts of the magnitude we might expect to occur for two of the central outcomes of the study—completion of training and earnings. For outcomes where the control group mean is 0.5, we will be able to detect effects of 7.6 percentage points for the sample in a single site and 13.2 percentage points for a subgroup comprising one-third of a single site sample.

Exhibit B-3 shows the MDEs for a single ISIS site sample of 960 (which is 80 percent response of the full sample of 1,200 participants) and for subgroups of the stated size. The MDEs presented are the minimum detectable differences in outcomes (in percentage points) between two randomly assigned groups with 80 percent power when we perform a two-sided statistical test at 10 percent level of significance, assuming a regression R² of 0.10and no finite population correction. The differences are shown for various average outcome levels for the control group.

The last column of Exhibit B-3 shows that for a mean control group outcome of 0.5, the MDE for a single site is 7.6 percentage points. This means that if the true effect of the career pathways intervention is to raise the completion rate for a certificate (or for a gatekeeper college course) from 50 percent to 57.6 percent, we would have an 80 percent likelihood of obtaining an impact estimate that is statistically significant. If the true effect is less than 7.6 percentage points, there is a lower likelihood that differences between assignment groups will be statistically significant, though many might still be detected.

Exhibit B-3. Minimum Detectable Effects for Training Outcomes and other Prevalence Estimates for a Single ISIS Site and for Subgroups at a Single Site

Sample	Expected Total Number of Individuals Randomly Assigned	Number of Completed Follow-up Survey Interviews	MDE if Mean Training Completion Rate (or other outcome) for the Control Group is:
Sample	Expected Total Number of Individuals Randomly Assigned	Number of Completed Follow-up Survey Interviews	80%	65%	50%
Entire Site	1,200	960	6.1 pp	7.2 pp	7.6 pp
67% Subgroup	800	640	7.5 pp	8.8 pp	9.3 pp
33% Subgroup	400	320	10.6 pp	12.4 pp	13.2 pp

Notes: (1) The MDE’s are based on calculations which assume that two-sided tests are used at the 10 percent significance level, treatment and control groups are of equal sizes, the desired power is 80 percent, and the regression R² is 0.10. (2) All MDE’s assume a 80% survey response rate, with no finite population correction.

Power Calculations for Earnings

Exhibit B-4 shows the MDEs for earnings impacts for the sample in a single site and for subgroups of that sample. These MDEs are based on the Year Two earnings outcomes from the Sectoral Employment Impact Study (Maguire et al., 2010), a study of low-income individuals who participated in occupational job training programs in three sites. In a single ISIS study site, the proposed design will be able to detect differences between treatment and control of $1,887 or more in annual earnings. Although it is by no means assured that the career pathways interventions tested in ISIS will produce an earnings impact of over $1,887 per year, impacts smaller than this amount would have limited potential to meaningfully increase families’ self-sufficiency and hence may not be important to detect with high confidence.

Exhibit B-4.
Minimum Detectable Effects for Annual Earnings Impacts for a Single ISIS Site and for Subgroups at a Single Site

Sample	Expected Total Number of Families Randomly Assigned	MDE (dollars)
Sample	Expected Total Number of Families Randomly Assigned	MDE (dollars)
All Sites	1,200	1,887
67% Subgroup	800	2,311
33% Subgroup	400	3,268

Notes: (1) The MDE’s are based on calculations which assume that two-sided tests are used at the 10 percent significance level, the desired power is 80 percent, and the regression R² is identical to the Sectoral Employment Impact Study earnings regression (R² =0.142). (2) All MDE’s assume a 100% administrative record collection rate, with no finite population correction. (3) The variance of earnings is derived from the standard error of the ITT impact estimate for the experimental group (n≈470) vs. the treatment group (n≈513) in the Sectoral Employment Impact Study (Year 2): $838.

B.2.4 Who Will Collect the Information and How It Will Be Done

ISIS will enroll program participants on a rolling basis over a 2-year period from Fall 2011 to Fall 2013. Partner sites will recruit potential participants through their existing recruitment practices. Partner sites will include a short description of the ISIS program along with their existing recruiting materials. All recruitment materials will specifically describe that participants will be involved in a research study and that there are limited slots in the program.

Because of the program level differences, different sites will enroll participants in the study and collect information in different ways. Sites will conduct orientation and information sessions in either in a group setting or with individuals. However, all sites will follow these general steps for data collection:

Conduct an Orientation and Information Session
Obtain Informed Consent
Administer the BIF and SAQ
Perform Random Assignment

Intake Materials

Abt Associates will pre-package intake materials and send them to sites. A package will consist of a numbered envelope containing the following materials: two copies of the informed consent, one BIF, and one SAQ. The envelope, BIF, SAQ, and informed consent will all be pre-printed to have the same unique identification number on them. The BIF, SAQ, and informed consent also will have the OMB control number printed on every page.

Individual Intake

If the site is performing intake a single person at a time, the intake worker will take out a new materials packet and begin the process by giving the applicant a short description of the program in the program office. The intake worker will give a clear and engaging description of the opportunity to participate, the importance of the study, and what participation entails, and participants will be given an opportunity to ask questions. The session will provide enough information to make sure that participants are well-informed and know that they are being asked to volunteer freely. Those who agree to participate in the ISIS study will be asked to read and sign an informed consent agreement explaining the project. A copy of the informed consent form is attached as Appendix B. The intake worker will take the signed copy of the informed consent for ISIS records and give the applicant a separate copy for his/her own records. This process should take around 15 minutes from the start of the program description to the signing of the consent form.

After the informed consent process is complete, the intake worker will explain to the applicant how the rest of intake will work. The intake worker will explain to the applicant that they will start by asking them some basic demographic questions. At this point, the intake worker will open the MIS on the intake computer and begin asking the applicant questions from the BIF. The intake worker will put the applicant’s information directly into the MIS. Inputting the BIF should take around 15 minutes to complete.

After completing the BIF, the intake worker will give the applicant a copy of the SAQ and the envelope. The intake worker will explain for the second time that the SAQ contains emotional and psychosocial questions, and the intake worker will explain that the applicant can refuse to answer any question. The intake worker will reassert that the information on the SAQ is private, and no one from the program will see the answers. After the intake worker has answered any of the applicant’s questions, the intake worker will direct the applicant to a semi-private room to complete the SAQ. The intake worker will tell the applicant where he/she can be found in case the applicant has any additional questions. The intake worker will instruct the applicant to return the completed SAQ in the sealed envelope when they are finished. The intake worker should tell the applicant that it will take about 20 minutes to complete the form.

The intake worker will have the option of performing random assignment after the participant is finished with the SAQ. To complete random assignment, the intake worker will check boxes on the MIS indicating the applicant has signed the consent form and turned in the sealed envelope with the SAQ. After checking all required boxes, the intake worker will press the assignment button and outline the next step for the applicant depending on his/her assignment status.

Group Intake

Group intake sessions will begin with a check-in. Applicants will sign in with intake workers, and intake workers will provide each applicant with a packet of materials. Once intake workers have checked in all the applicants, an intake worker will give a description of the program and explain the informed consent process. Applicants will be given the opportunity to ask questions so they clearly know what participation in the study involves. After the explanation, the intake worker will instruct applicants to remove the two copies of the informed consent from the folder, read it over, ask any questions, and sign it. The intake worker will confirm that all applicants have signed the informed consent with a verbal cue (like, “Does anyone need more time?”). After all applicants have signed the consent, the intake worker will instruct applicants to remove the BIF from the envelope. The BIF will have a clearly marked cover page so there is no confusion as to which form is which. The intake worker will give applicants about 15 minutes to complete the BIF then will prompt the group to see if any applicants need more time.

Once everyone has finished the BIF, the intake worker will instruct applicants to remove the SAQ from the envelope and fill it out. Again, the SAQ will have very clear markings on the front cover to differentiate it from the BIF. Additionally, the SAQ will have clear instructions to seal it in the envelope once applicants have completed it. The intake worker will tell applicants that the form should take about 20 minutes to complete and that applicants can approach the intake worker to ask any questions about the form.

As applicants start finishing the form, the intake worker will go to the program computer with the ISIS MIS and begin to check out applicants. During check-out, the applicant will hand the intake worker the signed copy of the informed consent, a completed BIF, and a sealed envelope with the SAQ. It is imperative that the intake worker checks to make sure the unique ID on the BIF and the envelope match. The intake worker then will spot check the BIF to make sure it is complete and will input the required fields into the MIS and have the opportunity to assign the applicant. Not all sites will assign the applicant on the spot, but where this is the case, the intake worker should be prepared to explain the next step in the process to the applicant regardless of assignment status.

We estimate that intake activities associated with ISIS will take participants approximately 45 minutes to complete, including informed consent (10 minutes), the BIF (15 minutes), and the SAQ (20 minutes). These estimates are included in the burden calculations in Section A.12.

Check Out

After the intake process is complete, the intake worker will staple the informed consent to the BIF and lock both in a file cabinet. The intake worker will lock the envelopes containing the SAQs in a separate file cabinet. Only site staff who have completed the ISIS trainings for human subjects research protections and data security will have access to the file cabinets. Sites will use FedEx overnight to send all intake forms to Abt Associates on a bi-weekly basis. Sites will not keep any physical or electronic records of data collected in the baseline forms.

Training and Monitoring

Abt Associates will ensure that ACF provided confidentiality assurances are met through training and site monitoring procedures. The ISIS team (Abt staff and subcontractors) will train site staff in:

the appropriate language to use during recruitment about the random assignment nature of the study;
how to explain the study during intake and informed consent;
and how to appropriately safeguard data.

Abt Associates has set up site teams consisting of at least one senior and one junior staff member for each potential partnering site. The site team will be responsible for training partnering site staff to perform baseline data collection activities. ISIS trainers will use a training presentation developed with the assistance of the Abt Associates’ Institutional Review Board to train site staff on human research protections and data security. At the completion of the training sessions, ISIS trainers will have site staff sign a confidentiality agreement containing an Individual Investigator Agreement, confirming the staff is aware of all research guidelines. Initial trainings of site staff will occur in person immediately prior to the pilot phase of the study. In the case of staff turnover, new site staff will be trained using a recorded webinar of the same training module used at the start of the evaluation.

The training module will also include an explanation of the support materials created by Abt Associates to assist partner site staff. Training materials will include a site-specific procedural manual that explains the intake procedure, random assignment, and data collection. As part of the initial training, the ISIS trainer will walk through the training manual with site staff. Site staff will also be able to access additional support tools via the ISIS website or contact their ISIS site team with other questions.

The ISIS team will monitor recruitment, intake, and data security through regular technical assistance calls (likely bi-weekly) and site visits. During these calls, ISIS team members will probe partnering site staff for problems that have come up during the intake process. These probes will have the dual purpose of troubleshooting problems and making sure site staff is properly administering intake procedures. The ISIS site teams will visit each site at least once prior to the pilot period and once during the pilot period or early in the full implementation period to monitor their intake procedures. The frequency of site visits beyond this time is still being determined.

B.2.5 Procedures with Special Populations

The informed consent agreement, BIF, and SAQ will be available be in English and Spanish. To ensure participants can understand each of the documents, they are designed at a 9th-grade readability level. The ISIS team will work with sites on ways staff can assist where other translation assistance may be needed. Should site planning indicate likely difficulty, the ISIS team will provide translated materials.

B.3 Methods to Maximize Response Rates

All individuals who agree to participate in the evaluation must complete the BIF and the SAQ¹ in order to have the opportunity to be randomly assigned to the career pathways program. Therefore, a response rate of 100 percent is expected for these instruments.

B.4 Tests of Procedures

The ISIS team worked with the Community College of Baltimore County to recruit six individuals enrolled in the Essential Skills course for a pre-test of the English baseline forms (a separate pre-test of the Spanish forms is scheduled for September 20, 2011 with three students from Montgomery College). The students in this class generally have between an 8^th and 11^th grade reading level. As such, the class and educational background of the students is roughly equivalent to what we expect to find in Adult Basic Education classes that will be a part of some ISIS career pathways programs.

The volunteers were composed of a mix of four women and two men; three individuals were African American, and three were white; volunteers ranged in age from 19 to 39 years of age; and all of the volunteers were native English speakers. For the pre-test, we asked the volunteers to go through ISIS intake procedures as if they were in the study. After completing the forms, we engaged volunteers in 15-minute discussions about their experience.

Overall, volunteers did not have difficulty with any of the forms. While volunteers had some minor problems with individual questions, they largely understood the process. None of the volunteers had a problem with the layout of the forms. They thought printing the forms on different color paper was helpful and did not have trouble with the SAQ being printed and stapled horizontally. The volunteers thought questions were spaced appropriately and the font was large enough and clear enough to read without trouble. In a few cases, volunteers did fail to notice instructions or skip patterns for specific questions and suggested putting those instructions in bold font.

B.4.1 Revised Burden Estimates

Exhibit B-5 shows the time volunteers spent completing each form. We found that our estimate for how long it would take to complete the baseline forms was overstated. Our estimate for the BIF (15 minutes) was accurate (an average of 13 minutes to complete). However, our estimate for how long it would take to complete the SAQ was high. We estimated it would take 30 minutes to complete the SAQ, but the average time to complete was 15 minutes. Even though it took, on average, only 15 minutes to complete the SAQ, we suggest keeping the estimate to complete the form slightly higher because we anticipate some participants in the actual study having difficulty with some of the questions. We suggest revising our burden estimates listed in the OMB package to 15 minutes for the BIF and 20 minutes for the SAQ.

Exhibit B-5.

Pre-Test Time to Complete

	Time to Complete BIF	Time to Complete SAQ	Total
Volunteer 1	11	15	26
Volunteer 2	N/A^*	10	N/A^*
Volunteer 3	12	15	27
Volunteer 4	11	15	26
Volunteer 5	17	17	34
Volunteer 6	12	17	29
Average	13	15	28

^* This volunteer did not fill out a large amount of information so we discounted her time.

B.4.2 BIF

Volunteers had no significant problems filling out the BIF, although some stated they were uncomfortable answering some of the questions, and others had difficulty framing answers. However, all but one volunteer attempted to answer even the questions they found to be sensitive. Below we describe the areas where volunteers had the most questions.

Alternate Contacts

This section caused volunteers the most difficulty. Most did not know the addresses or phone numbers for up to three alternate contacts. Because this information is critical for the follow-up survey efforts, the ISIS team will advise ISIS partner sites on how to encourage potential participants to have this information available at intake (e.g., building a request to bring the information into the recruitment materials).

Employment, Earnings,and Other Sources of Income

The questions about employment and earnings caused several difficulties. One cause of difficulty was the order of questions and the answer categories. Specifically, question Q25 asks the date when the respondent last worked (month, day, and year of the last spell of employment). Some of the volunteers took a long time to remember such a precise date. We changed the response space to ask for the month and year only, which should be easier for respondents to remember.

A couple of volunteers were confused by the way we categorized sources of income. We listed welfare and public assistance in a separate category from food stamps and WIC, in that order. Some volunteers were receiving food stamps and categorized them as public assistance before seeing the separate category for WIC and food stamps. We changed the question order to have WIC and food stamps listed before welfare and public assistance. We added the phrase “not including WIC or food stamps” to the welfare and public assistance category. One participant was not sure if she should include non-court ordered child support. We included a phrase saying “official or unofficial” to include both categories. We also included an additional category asking if the respondent has received a housing choice voucher or public housing.

In our team review of the forms, we also noticed an inconsistency in wording with the question asking about total household income. The question asks for the family’s total household income, but family and household are distinct concepts. We have changed the wording to say: Last year, what was your total family income? Include your own earnings and any income from your spouse or partner and any other relatives who live with you.

Other Items of Interest

While it did not come up in discussions with the pre-test volunteers, the ISIS team noticed a potential problem with the education questions. The response categories for level of education are different for three questions. Two questions include the categories “Associate’s Degree (For example AA, AS)” and “Bachelor’s Degree or Higher (For example BA, BS),” while another question includes the responses “Two-Year College Degree,” “Four Year College Degree,” and “Graduate (Masters or Doctoral) Degree.” The ISIS team standardized categories to all say “Associate’s Degree (For example AA, AS)” and “Bachelor’s Degree or Higher (For example BA, BS).”

B.4.2 SAQ

No major revisions are necessary for the SAQ. As discussed below, some volunteers described various questions or sets of questions as “uncomfortable”, but on the whole, a clear majority thought the SAQ was easier to complete than the BIF.

Sensitive Questions

Some participants thought the SAQ questions were personal and made them uncomfortable. One volunteer said some of the questions made her “squirm in her seat.” Another was surprised that there were so many questions asking her about her feelings. We added a sentence to the instructions on the front cover of the form to prepare participants for the emotionally oriented questions. Even though there are some very sensitive questions about criminal history and domestic violence, none of the volunteers refused to answer questions systematically.

Category Description

The volunteers had no problems with the several questions we thought would potentially cause some difficulty (Appendix B). There is one question that asks respondents to distinguish between having just enough, some, or more than enough money. During the debrief each of the volunteers was able to clearly articulate the distinction between categories, and their descriptions were consistent. They stated just enough money meant being able to pay necessary bills, some money meant being able to make some extra purchases but having to make a decision on what additional purchases to make, and more than enough money meant being able to make purchases without tradeoffs. We also thought participants might have trouble answering different question batteries that have different time frames of reference (one set asks about the last week, another about the last month, and another about the last 12 months). Each of the volunteers said the directions were clear and they were able to answer each section without a problem.

‘Car’ Versus ‘Transportation’

We found one instance of a question providing information that we were not expecting, although it was not due to volunteers having trouble with the question. There is a question asking if the respondent has a car and another asking how often they have transportation problems. In this very small sample size, we noticed those with a car were just as likely to report transportation problems. Because we are concerned with transportation presenting a barrier to training or employment, we will likely capture the information of interest with the second question. We will consider dropping the question about possessing a car.

B.5 Individuals Consulted on Statistical Aspects of the Design

The individuals shown in Exhibit B-5 assisted ACF in the statistical design of the evaluation.

Exhibit B-6.
Individuals Consulted on the Study Design

Name	Telephone Number	Role in Study
Karen Gardiner Abt Associates Inc.	301-347-5116	Project Director
Daniel Kitrosser Abt Associates Inc.	301-347-5935	Deputy Project Director
Dr. Howard Rolston Abt Associates Inc.	301-634-1820	Principal Investigator
Dr. David Fein Abt Associates Inc.	301-634-1851	Principal Investigator
Dr. Larry Orr Consultant to Abt Associates Inc.	301-467-1234	Project Advisor

Inquiries regarding the statistical aspects of the study's planned analysis should be directed to:

Dr. Howard Rolston Principal Investigator Telephone: 301-634-1820

Dr. David Fein Principal Investigator Telephone: 301-634-1851

References

ACT, Inc. (2007). Talent Assessment: User and Technical Guide. Iowa City, IA.

Allen, J., Robbins, S., & Sawyer, R. (2010). Can measuring psychosocial factors promote college success? Applied Measurement in Education, 23(1), 1-22.

Bound, J., Lovenheim, M. F., & Turner, S. (2010a). Increasing Time to Baccalaureate Degree in the United States. National Bureau of Economic Research, Inc, NBER Working Papers: 15892. Retrieved from http://www.nber.org/papers/w15892.pdf.

Carnevale, A., Fry, R., & Lowell, B. (2001). Understanding, speaking, reading, writing, and earnings in the immigrant labor market. American Economic Review, 91(2), 159-163.

Chaney, D., Hammond, M., Betz, N., & Multon, K. (2007). The reliability and factor structure of the Career Decision Self-efficacy Scale-SF with African Americans. Journal of Career Assessment, 15, 194-105.

Choitz, V., Soares, L., & Pleasants, R. (2010). A new national approach to career navigation for working learners. Washington, DC: Center for American Progress.

Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health & Social Behavior, 24(4), 385-396.

Colquitt, J., LePine, J., & Noe, R. (2000). Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85(5), 678-707.

Conger, R., Rueter, M., & Elder Jr, G. (1999). Couple resilience to economic pressure. Journal of Personality & Social Psychology, 76(1), 54-71.

Conley, D. (2007). Toward a more comprehensive conception of college readiness. Eugene, OR: Educational Policy Improvement Center.

Crew Jr, R., & Davis, B. (2006). Substance abuse as a barrier to employment of welfare recipients. Journal of Policy Practice, 5(4), 69-82.

Cutrona, C., & Russell, D. (1987). The provisions of social relationships and adaptation to stress. Advances in Personal Relationships, 1.

Cutrona, C., Russell, D., Brown, P., Hessling, R., Clark, L., & Garder, K. (2005). Neighborhood context, personality, and stressful life events as predictors of depression among African American women. Journal of Abnormal Psychology, 114(1), 3-15.

Danziger, S., Kalil, A., & Anderson, N. (2000). Human capital, physical health, and mental health of welfare recipients: co-occurrence and correlates. Journal of Social Issues, 56(4).

Darity Jr, W., & Mason, P. (1998). Evidence on discrimination in employment: Codes of color, codes of gender. Journal of Economic Perspectives, 12(2), 63-90.

Engstrom, C., & Tinto, V. (2008). Learning better together: the impact of learning communities on the persistence of low-income students. Opportunity Matters, 1.

Estrada, R. (2010). How to build bridge programs that fit into a career pathway. Chicago, IL: Instituto del Progreso Latino.

Fein, D., & Beecroft, E. (2006). College as a job advancement strategy: final report on the New Visions Self-Sufficiency and Lifelong Learning Project. Bethesda, MD: Abt Associates Inc.

Goldrick-Rab, S., & Sorensen, K. (2010). Unmarried parents in college. Future of Children, 20(2), 179-203.

Gottschalk, P. (2005a). Can work alter welfare recipients' beliefs? [Article]. Journal of Policy Analysis & Management, 24(3), 485-498.

Hamilton, G., Brock, T., Farrell, M., Friedlander, D., & Harknett, K. (1997). Evaluating two welfare-to-work program approaches: two-year findings on the labor force attachment and human capital development programs in three sites. New York, NY: MDRC.

Hamilton, G., Freedman, S., Gennetian, L., Michalopoulos, C., Walter, J., Adams-Ciardullo, D., . . . Ahluwalia, S. (2001). How effective are different welfare-to-work approaches? Five-year adult and child impacts for eleven programs. New York, NY: MDRC.

Heckman, J. (1998). Detecting discrimination. Journal of Economic Perspectives, 12(2), 101-116.

Heckman, J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411-482.

Hinckley, R., & Hull, D. (2007). Career-limited adults in the era of high technology and globalization. In D. Hull & R. Hinckley (Eds.), Adult Career Pathways: Providing a Second Chance in Public Education. Waco, TX: CORD.

Jenkins, D. (2006). Career pathways: aligning public resources to support individual and regional economic advancement in the knowledge economy. New York, NY: Workforce Strategy Center.

Kessler, R. (2002). Epidemiology of depression. In I. Gotlib & C. Hammen (Eds.), Handbook of Depression. New York, NY: The Guilford Press.

Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college outcomes: constructing the Student Readiness Inventory. Educational & Psychological Measurement, 65(3), 482-508.

Lee, S., & Vinokur, A. (2007). Work barriers in the context of pathways to the employment of welfare-to-work clients. American Journal of Community Psychology, 40(3/4), 301-312.

Leininger, L., & Kalil, A. (2008). Cognitive and non-cognitive predictors of success in adult education programs: evidence from experimental data with low-income welfare recipients. Journal of Policy Analysis & Management, 27(3), 521-535.

Maguire, S., Freely, J., Clymer, C., Conway, M., and & Schwartz, D. (2010). Tuning In to Local Labor Markets: Findings From the Sectoral Employment Impact Study. Philadelphia, PA: Public/Private Ventures.

Matus-Grossman, L., & Gooden, S. (2002). Students' perspectives on juggling work, family, and college. New York, NY: MDRC.

Michalopoulos, C., & Schwartz, C. (2001). What works best for whom: impacts of 20 welfare-to-work programs by subgroup. New York, NY: MDRC.

Morgenstern, J., Neighbors, C., Kuerbis, A., Riordan, A., Blanchard, K., McVeigh, K.H., Morgan, T.J., McCrady, B. (2009). Intensive Case Management Improves 24-Month Abstinence and Employment Outcomes for Substance Dependent Women Receiving Temporary Assistance to Needy Families (TANF). American Journal of Public Health, 99, 328-333.

Office of Management and Budget. (2006). Questions and answers when designing surveys for information collections. Washington, DC.

Olson, K., & Pavetti, L. (1996). Personal & family challenges to the successful transition from welfare to work. Washington, DC: The Urban Institute.

Peterson, C., Casillas, A., & Robbins, S. (2006). The Student Readiness Inventory and the big five: examining social desirability and college academic performance. Personality & Individual Differences, 41(4), 663-673.

Pleasants, R., & Clagett, M. (2010). Career pathways: background paper for a discussion of how the federal government can support their expansion. Boston, MA: Jobs for the Future.

Price, R., Choi, J., & Vinokur, A. (2002). Links in the chain of adversity following job loss: how financial strain and loss of personal control lead to depression, impaired functioning, and poor health. Journal of Occupational Health Psychology, 7(4), 302-312.

Purnell, R., & Blank, S. (2004). Support success: services that may help low-income students succeed in community college. New York, NY: MDRC.

Riger, S., Staggs, S., & Schewe, P. (2004). Intimate partner violence as an obstacle to employment among mothers affected by welfare reform. Journal of Social Issues, 60(4), 801-818.

Robbins, S., Allen, J., Casillas, A., Peterson, C., & Huy, L. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98(3), 598-616.

Robbins, S., Huy, L., Davis, D., Carlstrom, A., Lauver, K., & Langley, R. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130(2), 261-288.

Robbins, S., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: integrated meta-analytic path analyses. The Journal Of Applied Psychology, 94(5), 1163-1184.

Roderick, M., Nagaoka, J., & Coca, V. (2009). College readiness for all: the challenge for urban high schools. Future of Children, 19(1), 185-210.

Santor, D., & Coyne, J. (1997). Shortening the CES-D to improve its ability to detect cases of depression. Psychological Assessment, 9(3), 233.

Scrivener, S., Bloom, D., LeBlanc, A., Paxson, C., Rouse, C., & Sommo, C. (2008). A good start: two-year effects of a freshmen learning community program at Kingsborough Community College. New York, NY: MDRC.

Scott-Clayton, J. (2010). The Causal Effect of Federal Work-Study Assistance: Quasi-Experimental Evidence from West Virginia. Columbia University, Teachers College Working Paper. Retrieved from http://faculty.tc.columbia.edu/upload/js3676/ScottClayton_WorkStudy_08-04-2010.pdf.

Seefeldt, K., & Orzol, S. (2005). Watching the clock tick: factors associated with TANF accumulation. Social Work Research, 29(4), 215-229.

Seftor, N. S., & Turner, S. E. (2002). Back to School. [Article]. Journal of Human Resources, 37(2), 336-352.

Spitzer, R., Williams, J., Kroenke, K., Hornyak, R., & McMurray, J. (2000). Validity and utility of the PRIME-MD patient health questionnaire in assessment of 3000 obstetric-gynecologic patients: the PRIME-MD Patient Health Questionnaire Obstetrics-Gynecology Study. American Journal Of Obstetrics And Gynecology, 183(3), 759-769.

Stephens, R. (2009). Charting a path: an exploration of the statewide career pathway efforts in Arkansas, Kentucky, Oregon, Washington and Wisconsin. Seattle, WA: Seattle Jobs Initiative.

Turner, H., & Turner, R. (2005). Understanding variations in exposure to social stress. Health: An Interdisciplinary Journal for the Social Study of Health, Illness & Medicine, 9(2), 209-240.

Turner, R., & Avison, W. (2003). Status variations in stress exposure: implications for the interpretation of research on race, socioeconomic status, and gender. Journal of Health & Social Behavior, 44(4), 488-505.

Turner, R., & Lloyd, D. (1999). The stress process and the social distribution of depression. Journal of Health & Social Behavior, 40(4), 374-374.

Turner, R., Wheaton, B., & Lloyd, D. (1995). The epidemiology of social stress. American Sociological Review, 60(1), 104-125.

Vinokur, A., Price, R., & Caplan, R. (1996). Hard times and hurtful partners: how financial strain affects depression and relationship satisfaction of unemployed persons and their spouses. Journal of Personality & Social Psychology, 71(1), 166-179.

Vinokur, A., & Schul, Y. (2002). The web of coping resources and pathways to reemployment following a job loss. Journal of Occupational Health Psychology, 7(1), 68-83.

Vinokur, A., Schul, Y., Vuori, J., & Price, R. (2000). Two years after a job loss: long-term impact of the JOBS program on reemployment and mental health. Journal Of Occupational Health Psychology, 5(1), 32-47.

Washington State Board for Community and Technical Colleges. (2005). I-BEST: a program integrating adult basic education and workforce training. Olympia, WA.

Wlodkowski, R. J., Mauldin, J. E., & Campbell, S. (2002). Early Exit: Understanding Adult Attrition in Accelerated and Traditional Postsecondary Programs. Synopsis. July, Indianapolis: Lumina Foundation for Education, 2002

1 Sample members will have the option of skipping any item except a few key items on the BIF and skipping any item on the SAQ. Completion of the BIF therefore refers to completing the few key items that must be answered. Completion of the SAQ refers to returning the SAQ in the sealed envelope.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Basic Information Form
Author	NicholsonJ
File Modified	0000-00-00
File Created	2021-02-01