Attachment 11 - Registered Apprenticeship Hub Grantee Staff Interview Protocol_clean

OMB Control Number:1290-0NEW

OMB Expiration Date:

Apprenticeship Building America Portfolio

Implementation study of the ABA Category 4-Registered Apprenticeship Hub grants

Grant Administrator and Staff Discussion Guide

INTRODUCTION 

 

We are researchers with [The Urban Institute/Social Policy Research Associate}, a nonprofit research organization based in [Washington, DC/California] which conducts policy-related research on a variety of social welfare and economic issues. 

 

This project is being conducted under contract to the U.S. Department of Labor. Our discussion today is part of the Evaluating Registered Apprenticeship Initiative (ERAI), a national study sponsored by the U.S. Department of Labor to understand the Apprenticeship Building America (ABA) investments in apprenticeship. The study will surface new evidence on efforts to expand and diversify registered apprenticeships and to inform policymakers and practitioners on how these grants are helping to solve issues in workforce systems, particularly around barriers to employment for traditionally underrepresented populations. A major aim of the study is to learn more about the Registered Apprenticeship Hub grants, designed to support the establishment, scaling and expansion of Registered Apprenticeship programs. In each grantee we visit, we will be speaking with grant administrators and staff, program partners (including employers), and Technical Assistance customers.

Privacy Statement:   We know that you are busy, and we will be as focused as possible and will only ask questions that are relevant to your experience. We have many questions and will be talking to many different people, so please do not feel as though we expect you to be able to answer every question. Your participation in this interview is voluntary and you may choose not to answer some or any of our questions. We will do everything we can to secure the privacy of the information you provide but ask that you consider using headphones or finding a private space to talk so that other people cannot hear or see this conversation.

We believe the risks of participating in this study are minimal.

We believe there are no direct benefits to you, but we hope that the findings from this study will benefit the Apprenticeship Building America Portfolio project by providing research insights on how Hub grantees are helping employers design, develop and deliver programs and are supporting the expansion of apprenticeship programs.

My colleague and I will be taking notes in order to document what we hear during our discussion, and we may record this discussion. We will not share these notes with anyone outside of our research team, including the Department of Labor, and we will destroy these notes after the end of our project. When we compile our reports, the names of individual respondents will not be included. If we choose to quote you, you will only be identified by your title. You will not be quoted directly by name in any of our reports. While it is possible that you might be identified by your title, we will do our best to minimize the chance of that occurring. 

Finally, to help us accurately capture the information you share, we would like to record this interview. The recording is just a back-up for our notes and will be kept within our small research team. The interview recording will be deleted once we have developed a full set of notes from the interview.

 

Do you have any questions for me?  

[If we decide to record the interview] Are you okay with us recording the interview to improve the accuracy of our notes?  

Do I have your permission to begin the interview? [BEGIN INTERVIEW IF CONSENT GRANTED]

OMB Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays an Office of Management and Budget (OMB) control number. The valid OMB control number for this information collection is 1290-0NEW. The time required to complete this collection of information is estimated to average 60 minutes, including the time to review instructions, search existing data resources, gather the data needed and complete and review the collection of information. Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Chiefevaluationoffice@DOL.gov and reference the OMB Control Number 1290-0NEW. 

Note: Document the following information for each respondent:

• Name, title and organization/affiliation

• Role in the Hub and other organization roles

• Length of involvement with the organization

1. BASIC GRANT INFORMATION

1. Verify with the respondent the following background information about the grant. [Note: Collect and pre-fill this information prior to the visit from the grant application or other available documentation and verify with the respondent.]

1. Original grant period: ___________ to ___________

2. Modified grant period (if applicable): ___________ to ___________

3. Does your grant include the following types of apprenticeships?:

___Registered Apprenticeships

___Pre-apprenticeships

4. For registered/unregistered apprenticeship programs, please identify the following:

1. Industries targeted:

2. Occupations targeted:

5. For pre-apprenticeship program component, please identify the following:

3. Industries targeted:

4. Occupations targeted:

2. Have there been any modifications under your grant (e.g., scope of work, budget, performance period, etc.)? If yes, please briefly describe.

3. What are the goals of your Hub? Has it changed over time? If yes, how so?

4. Who do you consider the Hub’s customers? Employers, workforce organizations, secondary and postsecondary schools, others? [See survey response.} Why are you targeting them?

2. ENVIRONMENTAL CONTEXT

5. What geographic area(s) does your grant serve, including various partners (e.g., identify specific states, counties or regions served])?

[Note: Collect information prior to the visit and verify during interview.]

6. What is the economic environment and labor market demand in the geographic area(s) served for trained workers and employers in industries and occupations that are the focus of this grant? Has growth or shrinking of the labor market in target industries changed since the start of the grant?

[Probe for: growth industries, extent to which industries rely on apprenticeship; industries that are most likely to feature apprenticeships/pre-apprenticeships; economic conditions such as job openings in sectors/occupations that are focus of grant]

7. Please describe the needs of employers and workers in the local labor markets and industries served by this grant.

8. Is the grant program contributing to improvements meeting employer and workers’ needs in the local labor market? If yes, how so? If not, why not and what have been the challenges?

9. What was the extent to which registered apprenticeship and pre-apprenticeship opportunities were available, and in which industry sectors/occupations in the area(s) served, prior to the grant?

[Probe for: extent of linkages of grantee/partners to apprenticeships and linkage or overlap between this grant-funded project and other initiatives]

10. Are there any federal, state, or local policies governing apprenticeship programs in the geographic area(s) served that affect service delivery and implementation of grant activities, such as training opportunities or support services available?

[Probe for: policy context/environment]

11. Are there any other factors since the start of your grant that have affected your programs’ ability to recruit participants or partners or scale programming?

[Probe for: in- or out-migration of major employers, COVID-19, major layoffs, base closings, natural disasters]

3. GRANT ORGANIZATIONAL STRUCTURE

12. Please provide an overall description of the organizational structure of this grant. [Note: request organizational diagram if one is available.]

13. Does your organization function as an apprenticeship sponsor or work as part of a multi-partner network to sponsor apprenticeships? If not, which entities sponsor apprenticeship within your network of partners?

14. Who are the key staff at the grantee organization and what are their roles under the grant?

1. Number and position/title of staff funded under grant (including percent of effort for each grantee staff person)

2. Staff role performed under the grant

3. Prior experience of key staff with implementing apprenticeship programs

15. What is your organization’s previous experience with apprenticeships (prior to receipt of the grant)?

16. In your role as the grantee, what is your perception of your organization’s role in supporting apprenticeship expansion? What functions does your organization play (e.g., as a convener, apprenticeship sponsor, technical assistance provider, etc.)?

17. What key strategies have you utilized under the grant to implement the Hub model and support industry’s needs for creating and expanding new and emerging sectors and occupations? Strategies are described in more detail later in the interview.

[Probe for: strategies such as developing technical assistance resources, providing incentive funding, and partner engagement]

18. In the startup and planning phase of the grant, what worked well, and what changes would you make in the future?

19. To what extent did you coordinate across other ABA-funded Hubs projects and initiatives to share insights and peer learning? If coordination occurred, what were the benefits and challenges of this coordination?

4. KEY PARTNERS

Note: Prior to the virtual site visit, collect the following information from the grant application and survey on the following to the extent it is available:

1. Who are the collaborating partners (including employers) listed on the grant application?

[Provide a full list].

2. What methods did the proposal specify for reaching out to and involving partners (including employers)?

3. What roles, if any, did the grantee propose to play in helping expand apprenticeship in coordination with these partners?

20. Who are your key partners in this grant? For each organization, please discuss:

    1. name and type of the organization,

    2. amount of subgrant/funding,

    3. role and key services delivered under the grant as part of Hub partnership network,

[Probe for potential roles:

• Planning and design of Hub activities

• Registering RAPs

• Developing Pre-apprenticeship Programs

• Marketing and Outreach of RAPS/pre-As

• Developing curriculum and credentials

• Delivering related instruction

• Delivering technical assistance to customers/employers

• Engaging employers (other than through TA)

• Enabling access to supportive services for apprentices

• Providing connections to other workforce development initiatives

• Facilitating work-based learning opportunities (for pre-apprenticeship)

• Providing incentive funding to employers

• Employer partner

• Conducting employer outreach/engagement

• Helping with apprenticeship program or curriculum design

• Serving as an apprenticeship sponsor

• Serving as a pre-apprenticeship sponsor

• Serving as a related training instruction provider

• Providing support services for apprentices]

4. level of involvement,

5. whether partner is meeting expectations or are any challenges in engaging this organization, and

6. the benefits of the partnership.

21. Are there any partners not receiving grant funds who are considered key partnering organizations? What are their roles?

22. What was your organization’s process and approach to identifying and selecting key partners in the grant and what was their involvement in the grant planning process?

23. How did these partners collaborate during the grant to support rapid development, scaling and deployment of RAPs? What were the main ways in which they collaborated and to what end?

24. Did the grantee or partners experience start-up or early implementation issues (e.g., difficulties hiring staff, problems engaging partners, lack of employer interest/engagement, lack of potential participant interest, external (environmental) factors? Please discuss each challenge, including how it affected grant implementation, if and how the challenges were overcome.

25. How have partnerships evolved or changed over the course of the grant to date?

5. RAP EXPANSION STRATEGIES AND SYSTEMS CHANGE GOALS

26. Please describe any strategies used to engage industry and employers to expand apprenticeship through this grant and any changes over the course of the grant period.

27. We want to understand 1) whether your organization has utilized any of the following strategies over the course of the grant to date and 2) a description of how the strategy has been implemented as part of your grant project.

    1. Capacity building (e.g., increasing number of apprenticeship slots to improve pipeline of trained workers)

    2. Industry engagement (e.g., employer involvement in design of initiative and level of strategic partnership)

    3. Data-driven decision making (e.g., data collected, data infrastructure, data collection methods, and data sharing methods)

    4. Collaboration (e.g., gather input, negotiate with partners, build consensus)

    5. Quality and accessibility (e.g., monitoring quality to suggest improvements and determine replicability/scalability)

    6. Strategies to advance equity in apprenticeship

    7. Expansion strategies for increasing the number of programs, populations served, number of training slots, etc. (scale)

    8. Other (please describe)

28. Which changes or improvements to education and workforce systems can be expected during the grant period (including policy changes) as a result of adopting these strategies?

29. What were the challenges for implementing the strategies identified? Were there any protective factors or assets that made implementing certain strategies easier than others?

30. To date, what do you consider to be your most successful strategies implemented under the grant?

6. TECHNICAL ASSISTANCE SERVICES

31. How did implementing technical assistance services to support apprenticeship expansion align with the goals of your grant?

32. What technical assistance did the Hub offer to your customers (including employers) designed to help with apprenticeship expansion? What did this entail?

Probe for:

• Purpose of T/TA activity

• Content of activity

• Mechanism through which TA was delivered

• Coordination between organizations providing TA

• Number of orgs or groups that received TA

• Number of individual recipients

33. Did any of your partners offer technical assistance designed to help with apprenticeship expansion? What did this entail?

Probe for:

• Purpose of T/TA activity

• Content of activity

• Mechanism through which TA was delivered

• Coordination between organizations providing TA

• Number of orgs or groups that received TA

• Number of individual recipients

34. How did your customers respond to the technical assistance provided? Were there aspects they used more than others? Were there some aspects they didn’t find helpful?

35. Were any changes made over time to the implementation of technical assistance strategies over the course of the grant?

36. Were technical assistance resources provided by your organization or partners helpful for customers in the field? What feedback did you receive from them about usefulness of TA provided?

37. What are promising practices from your perspective as the grantee for providing technical assistance? What was not successful and why?

38. What progress has been made to date when it comes to provision of technical assistance resources and what remaining steps (if any) are to follow?

7. IDENTIFIED OUTPUTS AND OUTCOMES

39. What data management systems did you implement for the grant? What metrics and data did grantees, partners, and other stakeholders use to define and measure success? [Probe if they used others than the required performance measures for the grant.] How did you and your partners use the metrics and data for continuous improvement?

40. What progress to date has been accomplished toward your performance measures?

41. Please describe any challenges related to data infrastructure or data collection.

42. What has been the role of your partners in contributing data toward grant outcomes and performance measures? Have partners had a role in collecting and sharing data? If so, how/using what systems?

8. SUSTAINABILITY AND LESSONS LEARNED

43. Have you developed sustainability plans for the apprenticeships and partner engagement strategies implemented under the grant? If so, please describe these plans. What sources of funding will be used to sustain the apprenticeship activities implemented under the grant? Do you feel that your sustainability plan and leveraged resources are sufficient to sustain the activities of the grant after the completion of the grant?

44. What partnerships have been most successful? With which partners and why?

45. To what extent do you think your Hub program/initiative could and should be replicated in other localities? What features of your program are most amenable to replication? What features of the project are least amenable to replication? How does location, the target population served, or other distinctive features of your program make it either non-transferable or limit transferability?

46. How might additional funding or investment impact the goal of supporting industry needs and apprenticeship expansion supported by this grant?

47. To date, what do you believe to be the main lessons learned from your grant?

48. What innovations and strategies under the grant can contribute to efforts to encourage more employers to adopt apprenticeships in emerging sectors and occupations?

11