Supporting Statement Part A - PROMISE

Supporting Statement for the Promoting Readiness of Minors in SSI (PROMISE) Evaluation

December 10, 2013

Gina Livermore

Holly Matulewicz

Jacqueline Kauff

Contents

Supporting Statement for The PROMISE EVALUATION 1

Part A. Justification for the Study 3

A1. Circumstances Making the Collection of Information Necessary 3

A2. Purposes and Uses of the Information 4

A3. Use of Technology to Reduce Burden 7

A4. Efforts to Avoid Duplication 7

A5. Methods to Minimize Burden on Small Entities 7

A6. Consequences of Not Collecting Data 7

A7. Special Circumstances 8

A8. Federal Register Announcement and Consultation 8

A9. Payments or Gifts 9

A10. Assurances of Confidentiality 9

A11. Justification of Sensitive Information 10

A12. Estimates of Hours Burden 10

A13. Estimates of Cost Burden to Respondents 11

A14. Annualized Cost to the Federal Government 12

A15. Reasons for Program Changes or Adjustments 12

A16. Plans for Tabulation and Publication of Results 13

A17. Approval Not to Display of Expiration Date for OMB Approval 13

A18. Explanation of Exceptions 13

Part B. Collection of Information Employing Statistical Methods 15

B1. Respondent Universe and Sampling Methods 15

B2. Procedures for the Collection of Information 16

B3. Methods to Maximize Response Rates and Deal with Nonresponse 17

B4. Tests of Procedures or Methods to Be Undertaken 18

B5. Individuals consulted on Statistical Aspects of the Design and on Collection and/or Analyzing Data 18

References 19

ATTACHMENT a: SECTION 1110 [42 U.S.C. 1310] LEGISLATION

Attachment B: Staff Interview Topics

AttAchment C: Social network Surveys

Attachment D: Focus Group Protocols

Attachment E: Focus Group INVITATION LETTERs

Attachment F: Focus Group PHONE Recruitment script

Attachment G: Focus Group Reminder Letter

AttachMENT H: FEDERAL REGISTER NOTICES

Supporting Statement for The PROMISE Evaluation

A. Justification for the Study

A1. Introduction

The Promoting Readiness of Minors in SSI (PROMISE) demonstration pursues positive outcomes for children with disabilities who receive Supplemental Security Income (SSI), and their families, by reducing dependency on SSI. The Department of Education (ED) awarded six cooperative agreements to states to improve the provision and coordination of services and support for children with disabilities who receive SSI and their families to achieve improved education and employment outcomes. ED awarded PROMISE funds to five single-state projects and one six-state consortium.¹ With support from the Department of Labor (DOL) and the Department of Health and Human Services (HHS), the Social Security Administration (SSA) will evaluate the six PROMISE projects.  SSA contracted with Mathematica Policy Research to conduct the evaluation.

Under PROMISE, targeted outcomes for youth include: (1) an enhanced sense of self-determination; (2) achievement of secondary and post-secondary educational credentials, and attainment of early work experiences culminating with competitive employment in an integrated setting; and (3) long-term reduction in reliance on SSI. Outcomes of interest for families include: (1) heightened expectations for and support of the long-term self-sufficiency of their youth; (2) parent or guardian attainment of education and training credentials; and (3) increases in earnings and total income. To achieve these outcomes, we expect the PROMISE projects to make better use of existing resources by improving service coordination among multiple state and local agencies and programs.

SSA is requesting clearance for the collection of data needed to implement and evaluate PROMISE. The evaluation will provide empirical evidence on the impact of the intervention for youth and their families in several critical areas, including: (1) improved educational attainment; (2) increased employment skills, experience, and earnings; and (3) long-term reduction in use of public benefits. We will base the PROMISE evaluation on a rigorous design that will entail the random assignment of approximately 2,000 youth in each of the six projects to treatment or control groups (12,000 youth total). The PROMISE projects will provide enhanced services for youth in the treatment groups; whereas the youth in the control groups will only use those services already available in their communities independent of the interventions.

The evaluation will assess the effect of PROMISE services on educational attainment, employment, earnings, and reduced receipt of disability payments. The three components of this evaluation include:

• The process analysis, which will document program models; assess the relationships among the partner organizations; document whether the grantees implement the programs as planned; identify features of the programs that may account for their impacts on youth and families; and identify lessons for future programs with similar objectives.

• The impact analysis, which will determine whether youth and families in the treatment groups receive more services than their counterparts in the control groups. It will also determine whether treatment group members have better results than control group members with respect to the targeted outcomes noted above.

• The cost-benefit analysis will assess whether the benefits of PROMISE, including increases in employment and reductions in benefit receipt, are large enough to justify its costs. We will conduct this assessment from a range of perspectives, including those of the participants, state and federal governments, SSA, and society as a whole.

SSA is planning several data collection efforts for the evaluation. These include: (1) follow-up interviews with youth and their parents or guardians 18 months and five years after enrollment; (2) phone and in-person interviews with local program administrators, program supervisors, and service delivery staff at two points in time over the course of the demonstration; (3) two rounds of focus groups with participating youth in the treatment group; (4) two rounds of focus groups with parents or guardians of participating youth; and (5) collection of administrative data.

In this package, SSA only requests clearance for the interviews to be conducted with program staff and the focus group discussions to be conducted with youth and parents or guardians. SSA will request clearance for the 18-month and five-year survey interviews in a future submission.

A1. Authoring Laws/Circumstances Making the Collection of Information Necessary

Since 1980, Congress requires SSA to conduct demonstration and research projects to test the effectiveness of possible program changes that could encourage individuals to work and decrease their dependence on disability benefits. In fostering work efforts, SSA intends for this research and the program changes evaluated to produce federal program savings and improve program administration. Section 1110 of the Social Security Act authorizes SSA to conduct research and evaluation projects.

Youth who receive SSI face substantial barriers in making the transition to adult life. In addition to the issues facing all transition-age youth, SSI recipients and their families must consider issues related to their impairment and eligibility for continuing supports, especially cash assistance and medical insurance, as they move into young adulthood. SSI recipients who work and earn income above a certain threshold generally lose $1 of benefits for every $2 of earnings. Upon reaching age 18, child SSI recipients must undergo a redetermination of eligibility based on the adult definition of disability to continue receiving cash assistance. Uncertainty surrounding the outcome of that process may influence the decisions by youth to seek education, training, and work skills prior to age 18, as well as the support of families for their investment in human capital (Loprest and Wittenburg 2007). The poor outcomes of child SSI recipients prior to and after age 18 are indicative of the challenges they face moving into adulthood. Nearly one-third of them drop out of high school prior to age 18, and 43 percent have had problems in school that have resulted in their suspension or expulsion (Hemmeter et al. 2009). Relative to other young adults, former child SSI recipients after age 18 are more likely to be inactive in employment, school, and service programs; have higher rates of arrest; and have higher school dropout rates after age 18 (Wittenburg 2011; Hemmeter et al. 2009; Loprest and Wittenburg 2007). Approximately two-thirds continue to receive SSI as adults and only 22 percent work between the ages of 19 and 23 (Loprest and Wittenburg 2007). These poor outcomes may reflect the unique characteristics of these youth, particularly their severe impairments; however, they may also reflect factors associated with their families, such as low incomes, and other characteristics of the service environment.

A growing body of research suggests the importance of families in the employment outcomes of transition-age youth with disabilities. Previous studies demonstrated positive associations between the employment outcomes of these youth and the resources of their families, such as income, education, and family structure (Chiang et al. 2012; Emerson 2007; Loprest and Wittenburg 2007; Shattuck et al. 2012). Further evidence suggests that youth with disabilities rely primarily on family networks to find jobs (Hasazi et al. 1985) and they report family involvement as more important than other transition factors to their success (Powers et al. 2007). Family expectations about employment may be a particularly important determinant of the employment outcomes of transition-aged youth with disabilities (Blacher et al. 2010; Carter et al. 2012; Lee and Carter 2012; Lindstrom et al. 2011; Lindstrom et al. 2007; Simonsen and Neubert 2013) – potentially more important than income (Carter et al. 2012) or family structure (Lindstrom et al. 2007). Carter et al. (2012) suggest that family expectations are associated with youths’ paid employment experiences during school, and so may improve youths’ post-school employment outcomes. The importance of families in youth transitions may be amplified by the weakness of the transition service environment. High school students with disabilities may experience significant gaps in services and lack linkages to adult services. Many do not get information from their schools on how to access needed services. The U.S. Government Accountability Office (GAO) (2006) reports that youth with disabilities and their families often have difficulties identifying and learning how to ask for the accommodations they need to succeed in school and the workplace. Outside systems do not consistently provide these youth with the supports they need to achieve positive adult outcomes, especially in the critical areas of continuing education and employment. For example, only about one-quarter of secondary special education students, ages 17 or 18, have vocational rehabilitation (VR) counselors involved in their transition planning (Cameto et al. 2004). The problem of accessing supports is compounded by a lack of coordination between school- and adult-based services as youth leave secondary school (Luecking and Certo 2003; U.S. GAO 2006; Wittenburg et al. 2002).

The PROMISE projects are intended to address key limitations in the existing service system for youth with disabilities. By intervening early in the lives of these young people, at ages 14–16, the projects will engage the youth and their families well before critical decisions regarding the age 18 redetermination are upon them. We expect the required partnerships among the various state and federal agencies that serve youth with disabilities to result in improved integration of services and fewer dropped handoffs as youth move from one agency to another. And by requiring the programs to engage and serve families and provide youth with paid work experiences, the initiative is mandating the adoption of critical best practices in promoting the independence of youth with disabilities.

The Office of Management and Budget (OMB) proposed PROMISE as an interagency project between HHS, DOL, ED, and SSA. OMB requested that SSA conduct a rigorous evaluation of the PROMISE projects, focusing on key outcomes of interest, including reductions in SSI payments. The information collected by the evaluation contractor will be used to assess the effectiveness of the interventions that the individual PROMISE projects implement.

A2. Purposes and Uses of the Information

The evaluator will conduct separate project staff interviews and focus groups with youth participating in PROMISE and their parents or guardians for each of the PROMISE projects. SSA will use information from these activities to conduct a process analysis of PROMISE implementation, addressing the following questions:

• What were the PROMISE interventions like in practice, from the perspective of the PROMISE grantees, partner and project staff, and participants?

• What factors contributed to the PROMISE project design and the implementation experience observed across the PROMISE projects? What did it take to implement the program?

• Given what was learned about program impacts, what are the implications of the program implementation experience? What can be done better? What successes should be enhanced, and what problems can be avoided?

The sections below provide a description of the specific nature, purposes, and uses of the staff interviews and participant focus group:

1. Staff Interviews: SSA will use information from the PROMISE project staff interviews to (1) document the projects, the environments in which they are implemented, and the nature of existing services to youth and their families; (2) describe the interagency and other partnerships developed to implement PROMISE; and (3) assess the extent to which the projects adhered to their intended service delivery models. Specific issues to be addressed under each of these topics include:

• Documenting the program. What is the basic structure and logic model for each program? What is the service environment for program operations? How did grantees conduct participant outreach and enrollment? What are each program’s staffing structure and services? How were services implemented? How do grantees track participation? What do treatment families, program staff, and partners think of program services? What are the grantee’s plans and objectives for their evaluations?

• Partnership development, maintenance, and roles. How were potential partners identified and approached to participate in PROMISE? Who are the major and secondary partners? What are their roles? What is the nature of the relationships among the partner organizations? How do the partners communicate and collaborate and how has this changed over time? What are the contractual or other form of agreements between the grantee and its partners; between the partners and service providers? To what extent do agreements and other arrangements encourage the partners to work toward demonstration goals? To what extent do they discourage them from doing so?

• Fidelity of activities to program model. How closely do the programs adhere to their plans and logic models? In what ways do they use their logic models to guide services and track and manage inputs, outputs, and outcomes? How consistently are the models implemented at local sites? How do programs collect operations and service information and use it for management and evaluation purposes?

The evaluator will conduct two rounds of site visits for purposes of conducting in-person interviews with PROMISE project and partner agency staff. These visits will occur in summer 2014 and winter 2016. The evaluator will conduct the interviews with directors and administrators of the PROMISE project, and of the state agencies and other community partners participating in PROMISE; and with PROMISE project staff responsible for arranging and delivering PROMISE services to participants. For the five single-state projects, the evaluator will interview an average of 10 program directors an d managers and 20 service provider staff members at each project during the site visits. For the six-state consortium project, the evaluator will interview a total of 25 administrators and 45 service provider staff members during the 2014 and 2016 data collections. Some of these interviews will occur during site visits and some by phone. Examples of program directors and managers include the PROMISE project director and principal investigators, administrators of state government agencies that participate in PROMISE, and executive directors of non-governmental or community-based organizations that provide services to PROMISE participants and other youth or adults with disabilities. Examples of PROMISE project staff include recruiters, case managers, employment specialists, benefits counselors, vocational rehabilitation counselors, and educational instructors and coordinators. We show the topics the evaluator will address during these semi-structured interviews in Attachment B.

During the staff interviews, the evaluator will ask interviewees to complete a brief social network questionnaire. The evaluator will administer separate versions of the questionnaire to program managers or directors and project staff, tailored to their specific perspectives (as shown in Attachment C). We designed this brief questionnaire to assess the strength and capacity of organizational collaborations associated with PROMISE. We will pre-fill the questionnaire with the names of organizations collaborating with each PROMISE project and which the evaluation team knows prior to the site visit. The respondents will add additional organizations to the form as needed. We will use the results to conduct a network analysis to examine whether and to what extent stakeholders interacted with one another before the implementation of PROMISE and whether and how their interactions change with the implementation of PROMISE. The analysis will also provide a means of examining which stakeholders are relatively more active participants in the PROMISE collaborative. We may also use the data to create independent variables for use in multivariate analyses that will investigate the extent to which communication and collaboration between PROMISE stakeholders is associated with program effects.

2. Participant Focus Groups: The evaluator will use the focus group data to describe the experiences of parents or guardians and youth enrolled in PROMISE and to supplement other data collected and used in the process analysis. Information collected in the focus groups will support analyses related to two key questions:

• How are the PROMISE projects being implemented and operated?

• What are the short-term impacts of the projects on youth and their parents or guardians?

To address these questions, evaluators will convene focus group discussions that concentrate on key areas of interest for each group (youth and parents or guardians). For youth, these topics will include: (1) program enrollment; (2) case management; (3) education services; (4) employment preparation and support; and (5) other program services. For the parents or guardians, key topic areas include: (1) program enrollment; (2) services for youth; (3) services to promote parent or guardian involvement; (4) staff and peer support for parents or guardians; and (5) services for parents or guardians. We will use the findings to assess client satisfaction with the demonstration; identify which aspects of the demonstration may be more or less associated with participation outcomes; identify potential improvements to the demonstration approaches; and provide feedback to the PROMISE projects.

During the site visits the evaluator will conduct during summer 2014 and the winter 2016, each of the five single-state projects will conduct a pair of focus groups. This includes one group with youth who are enrolled in PROMISE and one with their parents or guardians. For the six-state consortium project, we will conduct three pairs of focus groups during each round. We will conduct the youth and parent or guardian focus groups separately, but concurrently. This will ensure that we represent the perspectives of both groups which are the focus of PROMISE services. The evaluator will conduct the groups in English only, and will make accommodations to facilitate the full inclusion of people with disabilities.

The evaluator will conduct the recruitment efforts and moderate the group discussions using standard protocols (Attachment D) to structure the 90-minute discussion and encourage participation. The evaluator will work with local PROMISE project staff to identify 50 youth (and their parents or guardians) at each project who would be interested in taking part in the groups. We will contact up to 50 treatment group families to recruit approximately 12 individuals to participate in each 90-minute focus group (expecting that approximately 10 will participate on the day of the group). Two to four weeks before the focus group, we will send an invitation letter (or email) to the parent or guardian for each household identified by the project staff (Attachment E). We anticipate that we will need telephone follow-up to secure participation of the target number of participants. Trained evaluator staff will follow-up with these family members; explain the purpose of the session; answer any questions and respond to any concerns; and invite them to participate in the focus group discussions. Solicitation calls for each focus group will continue until 12 youth and 12 parents or guardians agree to participate. Staff will use a recruitment script (Attachment F) to describe the purpose of the focus group and ask the parent or guardian, and the youth to participate. We will assure all of the parents or guardians and youth contacted that participation is voluntary and will not affect their eligibility for SSI or any other benefits they receive, either now or in the future. One week before the focus group, evaluators will send a reminder to each individual who agreed to participate (Attachment G) along with directions to and a map of the focus group location. The day before the focus group, evaluators will call participants to remind them of the focus group date, time, and location.

The respondents are PROMISE program staff, youth participants in the PROMISE program, and parents or guardians of the youth participants.

A3. Use of Technology to Reduce Burden

To the extent possible, we will send invitations to and reminders about the focus groups via e-mail. We will record staff interviews and focus group sessions digitally. Because the social network survey consists of only six or fewer questions, and because the respondents will be the same individuals who participate in the in-person interviews during site visits, it is most practical and least burdensome to collect the data on hard copy immediately during the in-person interview. The questionnaire is designed to be self-administered. Members of the evaluation team will be present to answer questions respondents might have about it. Transmission of the social network questionnaire to and from phone interviewees will occur electronically via fax or email.

A4. Efforts to Avoid Duplication

The staff interviews, social network survey, and focus group discussions will provide information that we cannot obtain through SSA’s administrative records. The discussions will be about the PROMISE-related experiences of staff and participants, the nature of partnerships and coordination with other agencies and programs, and the ways in which participation in PROMISE has affected participant educational and employment goals and experiences. Therefore, the nature of the information we are collecting and the manner in which we are collecting it preclude duplication. SSA does not use another collection instrument to obtain similar data.

A5. Methods to Minimize Burden on Small Entities

Some of the service providers we will interview may be staff of small entities. Our protocol will impose minimal burden on all organizations involved and discussions will be kept to one hour or less. We will hold the information we will request to the absolute minimum required for the intended use. Interviews will be scheduled at times that are convenient to the respondents. In this way, we will minimize the effect on small businesses and other small entities.

A6. Consequences of Not Collecting Data

Interviews and group discussions with local program administrators, program supervisors, and service delivery staff to support the process analysis will take place two times. The first visit will be soon after demonstration startup (summer 2014), and the second visit will occur after the programs have matured and gained experience providing services to participants (winter 2016). Two visits are necessary to develop an understanding of the intervention and steps taken to implement project services. The first visit will focus on start-up activities, the projects’ outstanding features, and key challenges. The second visit will assess how the projects evolved over time in response to their early experiences and the lessons learned about service delivery to SSI youth and their families. Fewer visits would not allow SSA to assess how the projects evolve over time to address significant challenges and leverage successes.

Focus groups with parents or guardians and youth will take place during the site visits conducted in summer 2014 and winter 2016. We will recruit independent groups of participants at each round to minimize burden and provide an opportunity to gather information from more PROMISE enrollees. Two rounds of focus groups are necessary to develop an understanding of the intervention and steps taken to implement project services as the projects evolve over time, and to capture changes in the experiences and outcomes of participants as they are served by the projects for a longer period.

Not collecting the information from the group discussions with project staff and the focus groups with program participants and their parents or guardians would result in a loss of qualitative data that could provide greater insights into the impact findings generated with the quantitative data collected via the surveys and administrative data. Conducting the groups at two points in time allows more time to elapse between the groups; more time for staff to provide services; and more time for PROMISE to affect the lives of participants. Further, by revisiting the same sites at two points in time, evaluation staff may be able to follow up on challenges observed early in the implementation period that implementation staff may resolve or improve between the visits. Finally, by speaking with youth and parents or guardians, as well as program implementation staff at each site, the evaluation will obtain a more balanced approach to understanding the implementation efforts than could be gained from interviewing implementation staff alone. Therefore, we cannot collect the information less frequently, or with fewer respondents.

A7. Special Circumstances

There are no special circumstances that would cause this information collection to be conducted in a manner inconsistent with 5 CFR 1320.5.

A8. Solicitation of Public Comment and other Consultation with the Public

The 60-day advance Federal Register Notice published on February 10, 2014, at 79 FR 7736, and we received no public comments. SSA published the second Notice on April 23, 2014 at 79 FR 22752. If we receive comments in response to the 30-day Notice, we will forward them to OMB.

As a first step in the PROMISE evaluation, SSA convened a technical advisory panel. The panel provided input on the evaluation criteria and research design. It consisted of researchers and advocates who reflected expertise in youth transition, disability, and evaluation design. The external experts were:

• Burt Barnow, PhD, George Washington University

• Hugh Berry, US Department of Education

• Mark Donovan, Marriott Foundation for People with Disabilities

• David Johnson, PhD, University of Minnesota

• Jamie Kendall, US Dept. of Health and Human Services

• Jeffrey Liebman PhD, Harvard University

• Pamela Loprest, PhD , The Urban Institute

An interdisciplinary team of economists, disability policy researchers, survey researchers, and information systems professionals on the staff of the evaluation contractor (Mathematica Policy Research and its subcontractor, BCT Partners) contributed to the design of the overall evaluation. These individuals include:

• Karen CyBulski, Mathematica

• Thomas Fraker, PhD, Mathematica

• Jacqueline Kauff, Mathematica

• Gina Livermore, PhD, Mathematica

• Holly Matulewicz, Mathematica

• Tonya Woodland, BCT Partners

Youth receiving SSI, and their parents or guardians, are the target audience for the participant focus groups. Through their involvement in these sessions, they will be able to provide first-hand feedback on their experiences with PROMISE. Findings from early groups will be used, where applicable, to the refine procedures and discussion topics for subsequent groups held at other sites, for the later round of focus groups, and for the 18- and 60-month questionnaires for which we will seek OMB approval in a separate submission. We have not, yet, consulted with any PROMISE participants, or program staff.

A9. Payments or Gifts

Program administrators or directors and PROMISE service provider staff will not be offered remuneration for completing interviews.

Each PROMISE focus group youth and parent or guardian participant will receive a $30 incentive in the form of a gift card, to express the study’s appreciation for their time. In addition to the gift card, evaluators will provide light refreshments and snacks during the focus group sessions. Such additional incentives are likely to increase the appeal of participation because they offset the burden for those who may attend the focus groups soon after their work or school day ends.

A10. Assurances of Confidentiality

The information provided during the staff interviews and focus groups is protected and held in confidential accordance with 42 U.S.C. 1306, 20 CFR 401 and 422, 5 U.S.C. 552 (Freedom of Information Act) 5 U.S.C. 552a (Privacy Act of 1974) and OMB Circular No. A-130. We will treat the data in a confidential manner unless otherwise compelled by law.

The study team takes seriously the ethical and legal obligations associated with the collection of confidential data. Ensuring the secure handling of confidential data is accomplished via several mechanisms, including obtaining suitability determinations for designated staff; training staff to recognize and handle sensitive data; protecting computer systems from access by staff without favorable suitability determinations; limiting the use of personally identifiable information in data; limiting access to secure data on a “need to know” basis and only for staff with favorable suitability determinations; and creating data extract files from which identifying information has been removed. We will make the assurances and limits of confidentiality clear in all advance materials sent to recruit potential participants and will restate them at the beginning of each focus group session. The Paperwork Reduction and Privacy Act statements will appear on the advance letter.

The PROMISE enrollment database will contain contact information that the evaluator will use to invite participants to the focus groups. Advance letters will provide assurance that information being gathered is for research purposes only. We will reiterate the same message at the start of each focus group session and participants will be asked to keep the focus group conversations confidential. We will not disclose the identity of the group participants to anyone outside of the evaluation team, and the information the participants provide which we will presented in public documents will not be attributable to specific individuals. The focus group facilitator will digitally record each focus group discussion beginning after all introductions have been made. We will inform participants about the recording and instruct that they may request that we suspend the recording at any time. If there are any objections to the recording, the facilitator will not record the session. We will not ask any identifying information during the focus group and moderators will only refer to group participants by their first names; thus no identifying information will be included in the digital recording.

We will require subcontractors, consultants, and vendors to establish confidential information safeguards that meet prime contract security requirements. The evaluation project director will take action to ensure that any confidential information provided to or generated by a subcontractor, consultant, or vendor is properly disposed of at the completion of the agreement between the parties.

All data collected during the interviews and focus groups will be destroyed in a secure manner at the completion of the evaluation.

A11. Justification of Sensitive Information

This information collection does not contain any questions of a sensitive nature.

A12. Estimates of Hours Burden

Table A.1 shows the expected number of participants in the qualitative data collections, number of interviews, hours per response, and the total response burden overall and by year.

Over the course of the evaluation, we will conduct a total of 440 staff interviews. Per staff member burden estimates for these interviews include time for setting up the interview appointment by phone or email (6 minutes) and participating in the interview (60 minutes).

For the focus groups, the estimated time per response varies from 5 minutes to review the advance letter and complete the telephone screening for focus group nonparticipants to 100 minutes for those who participate in the focus group (5 minutes for screening, 5 minutes for reminders by phone and mail, and 90 minutes for the group discussion). The respondents will spend the bulk of the annual burden time participating in the focus groups, which will last approximately 90 minutes. The estimated total annual burden for the focus groups is 320 hours per round. This includes contacting and screening up to 50 enrollees (to obtain 10 focus group participants and 40 nonparticipants) per round at each of the five single-state projects, and 150 enrollees (to obtain 30 focus group participants and 120 nonparticipants) per round at the six-state consortium project.

Table A.1. Estimated Total Annual Burden by Respondent Type

2014 Interviews and Focus Group Discussions:

Modality of Completion	Number of Responses	Frequency of Response	Average Burden Per Response (minutes)	Estimated Total Annual Burden (hours)
Staff Interviews with Administrators or Directors	75	1	66	83
Staff Interviews with PROMISE Project Staff	145	1	66	160
Youth Focus Groups – Non-participants	320	1	5	27
Youth Focus Groups – Participants	80	1	100	133
Parents or Guardian Focus Groups – Non-participants	320	1	5	27
Parents or Guardian Focus Groups – Participants	80	1	100	133
Totals	1,020			563

2016 Interviews and Focus Group Discussions:

Modality of Completion	Number of Responses	Frequency of Response	Average Burden Per Response (minutes)	Estimated Total Annual Burden (hours)
Staff Interviews with Administrators or Directors	75	1	66	83
Staff Interviews with PROMISE Project Staff	145	1	66	160
Youth Focus Groups – Non-participants	320	1	5	27
Youth Focus Groups – Participants	80	1	100	133
Parents or Guardian Focus Groups – Non-participants	320	1	5	27
Parents or Guardian Focus Groups – Participants	80	1	100	133
Totals	1,020			563

Grand Total:

Grand Total

2,040

1,126

A13. Estimates of Cost Burden to Respondents

There is no cost to PROMISE administrators or to service providers because they will be participating in the interviews as part of their paid work. There is no cost to youth; they are still engaged in the pursuit of secondary education and are assumed not to be wage earners. For parents or guardians, the cost burden is estimated using the average 2013 minimum wage rate across the states included in the evaluation (obtained from the U.S. Department of Labor website on state-by-state minimum wage data).² The total cost to parents or guardians for their time in this collection is shown in Table A.2. Costs for data collection, storage, processing, and other functions related to these data are born solely by the evaluation contractor.

Table A.2 Annual Cost to Respondents

2014 Annual Cost to Respondents:

Respondent Type	Number of Respondents	Frequency of Response	Average Burden Per Response (minutes)	Median Hourly Wage Rate (dollars)	Total Respondent Cost (dollars)
Parent or Guardian Focus Group – Non-Participants	320	1	5	$7.38	$196.01
Parent or Guardian Focus Group – Participants	80	1	100	$7.38	$984.20
Total	400				$1,180.21

2016 Annual Cost to Respondents:

Respondent Type	Number of Respondents	Frequency of Response	Average Burden Per Response (minutes)	Median Hourly Wage Rate (dollars)	Total Respondent Cost (dollars)
Parent or Guardian Focus Group – Non-Participants	320	1	5	$7.38	$196.01
Parent or Guardian Focus Group – Participants	80	1	100	$7.38	$984.20
Total	400				$1,180.21

Grand Total:

Grand Total

800

$2,360.42

A14. Annualized Cost to the Federal Government

The cost to SSA of conducting the PROMISE staff interviews and participant focus groups is $2,957,116. Table A.3 below shows the costs by year.

We budgeted labor costs by estimating the number of hours of required staff at the various wage levels, multiplying by the applicable wage rates, and multiplying the resulting subtotals by factors to cover fringe benefits and burden expense. The basis for estimating other direct costs varies with the type of cost estimated. We summed and multiplied the total of labor costs and other direct costs by a factor to cover general and administrative expenses, and included the fee.

Table A.3 Annual Costs to the Federal Government

Fiscal Year	Cost
2014	$33,953
2015	$1,279,610
2016	$895,518
2017	$748,035
Totals	$2,957,116

A15. Reasons for Program Changes or Adjustments

This is a new information collection which will increase the annual burden on the public.

A16. Plans for Tabulation and Publication of Results

With the PROMISE evaluation findings, SSA and ED will be able to advise federal policymakers and state administrators on the supports, services, policy, and program changes that could encourage individuals to work and decrease their dependence on disability and other public benefits. In fostering work efforts, the goal is to implement program changes which produce savings to the federal government and improve program administration.

The evaluator will analyze the information collected in the staff interviews and focus groups to prepare reports that contain the findings and their program and policy implications. We will not use complex quantitative analytical techniques with data from these collections. Four major reports will present the findings from the staff interviews and participant focus groups, as well as other information collected for the evaluation. The reports will include a stand-alone summary of the purpose, methodology, key findings, and policy implications, as well as a short executive summary. Products resulting from information obtained in this data collection will provide SSA with information about the experiences of PROMISE staff, staff of partner agencies, and participants. The evaluator will integrate the information obtained from the staff interviews and participant focus groups with information collected from the other components of the evaluation to draw comparisons across and within sites and describe factors that might explain any observed variation.

Table A.4 presents the planned timeline for the data collection, and the completion dates for the public reports that will include the interview and focus group findings.

Table A.4 Data Collection and Reporting Schedule

Activity/Report	Approximate Dates
Data Collection – Staff Interviews	Summer 2014 and Winter 2016
Data Collection – Youth and Parent or Guardian Focus Groups	Summer 2014 and Winter 2016
Reports – Early Assessment Reports	Spring 2015
Reports – Process Analysis Reports	Winter 2017
Reports – Final Services Impact Report	Summer 2018
Reports – Final Long-Term Evaluation Report	Winter 2022

A17. Approval Not to Display of Expiration Date for OMB Approval

SSA is not seeking an exemption with this submission. The OMB expiration date will be displayed on all focus group materials.

A18. Explanation of Exceptions

SSA is not requesting an exemption to certification requirements.

1 The six-state consortium project goes by the name Achieving Success by Promoting Readiness for Education and Employment (ASPIRE) rather than by PROMISE.

2 Data accessed from website on November 6, 2013. [http://www.dol.gov/whd/minwage/america.htm]