America’s Promise Site Visit Master Protocol MATHEMATICA POLICY RESEARCH
America’s Promise Jobs-Driven Grants Evaluation: Telephone Interview Protocol
Privacy and Taping
Good afternoon. My name is ___________________, and this is my colleague _________________ and we are from Mathematica Policy Research/Social Policy Research Associates. We are part of an independent research team that is studying the services being provided through [Program Name] as part of the America’s Promise Job Driven Grants. The study is on behalf of the U.S. Department of Labor. The department wants to learn about the efforts of regional partnerships to support job training in high-growth industries and whether such efforts help participants learn skills in careers that offer career growth and advancement. Thank you very much for agreeing to participate in this conversation. Your participation is very important to the study.
We value the information you will share with us and want to make sure we capture it all. We are recording our discussion today to help us remember what everyone said when we go back to write our report. Although we will be taping the session and taking notes, we will not use names or attribute information to any particular individual, and we will report our findings at an aggregate level. This recording is only listened to by our study team and will be erased at the conclusion of our study.
Before we begin, I just want to clarify that the America’s Promise program at this site is called [PROGRAM NAME]. If I reference “America’s Promise” at any point in the interview, I am referring to [PROGRAM NAME].
Are you okay with our recording the conversation? [If yes, start recorder. If no, take hand written notes.]
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 120 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Labor, Office of the Chief Information Officer, Attention: Departmental Clearance Officer, 200 Constitution Avenue, N.W., Room N-1301, Washington, DC 20210 or email DOL_PRA_PUBLIC@dol.gov and reference the OMB Control Number 12XX-XXXX. Note: Please do not return the completed form to this address. |
Name and title:
Name and type of company/organization:
How long have you been employed at the company/organization?
Briefly, can you describe your prior work experience?
Broadly speaking, what is your role in the [PROGRAM NAME]?
Can you describe in your own words the goal of [PROGRAM NAME]?
We know from our discussion prior to the [VISIT/CALL] that your target industries and occupations include [ADD DETAIL]. How and why were these industries and occupations selected? (RQ4)
Were others considered but not included? Why? (RQ4)
Have they changed over time? If so, why and how? (RQ4)
Describe the demand for careers in selected industries and the types of employers in the local area. What occupations are in high demand? Why? (RQ4, RQ5)
To what extent has the local economic context helped the [PROGRAM NAME] planning and implementation efforts? (RQ3, RQ5)
To what extent has the local economic context hindered the planning and implementation efforts? (RQ3, RQ5)
How is the region defined for the [PROGRAM NAME] grant? (RQ1)
How does [PROGRAM NAME] fit in with other regional workforce development initiatives and priorities? Does it build off of any previous initiatives? (RQ2)
Based on our discussions before the [VISIT/CALL], our understanding is that the [PROGRAM NAME] partnership includes [LIST organizations]. Were any of these partners added since your grant application? If so, which ones and why? (RQ7, RQ11)
Are any partners that were included in the grant application no longer involved? If so, which ones and why? (RQ11, RQ12)
Broadly speaking, what role does each partner play in the [PROGRAM NAME] grant? Probe: planning or oversight of grant activities? Service delivery? Job development? (RQ6)
How did the partnership for this grant form? (RQ7)
Was there a champion for the grant application? (RQ6, RQ10)
Who determined which specific organizations to include? Was the decision driven by grant requirements? What other factors were involved in the decision? (RQ7)
Were organizations approached that chose not to participate? Which ones and why? (RQ7)
Which grant partners have you partnered with in the past? Please briefly describe your past experience. In what capacity have you worked together? (RQ7)
Did the pre-existing partnership(s) involve activities similar to those being implemented through this grant? (RQ2)
Are any of the partnerships new? If so, which and how did you identify this partner and develop this partnership? (RQ7)
How many employers are engaged? How many did you approach to engage the ultimate group of employer partners? If any chose not to participate, what were the reasons? (RQ7)
How did you decide which employers to engage? And how were they identified? (RQ7)
What makes an employer a good candidate to partner with [PROGRAM NAME]? What qualities do you look for to determine which employers to partner with more? (RQ13, RQ14)
How did you approach the employers? Who was responsible? What was the messaging? (RQ7)
How have you worked with employer associations or a chamber of commerce to reach employers? (RQ7)
How does the grantee organization facilitate communication among partner organizations? (RQ8)
Does your organization participate in regular partner meetings? How frequently do they occur? What form do they take (in-person, conference call, etc)? (RQ8, RQ9)
Which partners participate in regular meetings? (RQ8, RQ9)
What are common topics for these meetings? (RQ16)
What other forms of communication does your organization use as part of the [PROGRAM NAME]? (RQ8)
How has this changed over time? (RQ11)
Which partners do you view as most critical for this effort? (RQ10)
What systems (workforce development system, higher education system, and economic development systems) do you perceive as critical to support and implement regional workforce and economic development efforts like [PROGRAM NAME]? (RQ6)
From your perspective, are the partners involved in [PROGRAM NAME] perceived as the right mix of partners? If not, what types of partners are missing? (RQ10)
Thinking specifically about employer and industry partners, how do you solicit their feedback? How do you use that feedback for continuous improvement in partnering with them and serving their needs? (RQ8)
Did your organization experience any challenges in relation to partnerships? For example, did you experience challenges recruiting, retaining, or engaging partners? With communication with or between partners? If so, describe. (RQ12)
How did the grantee and/or your organization address these challenges? (RQ12)
How does your organization define successful partner involvement as it relates to [PROGRAM NAME]? What do you perceive as the most successful aspects of your partnerships? (RQ13)
A. Eligibility, recruitment, and application
Who is your target population for [PROGRAM NAME] services? (RQ23)
Based on our understanding from your application, you were planning to serve [#] participants over the life of the grant. Is that right? (RQ15)
What proportion of those participants do you anticipate being unemployed? Underemployed? Incumbent workers? (RQ23)
Are there any other subpopulations that your region is targeting within those larger categories (e.g. veterans, groups underrepresented in the industry)? If so, what are they? How many individuals do you anticipate enrolling from each? (RQ23)
How did you determine which populations to target? (RQ23)
How are individuals recruited to participate in [PROGRAM NAME]? (RQ24)
Do you conduct outreach to inform potentially eligible applicants about the services available through [PROGRAM NAME]? What approaches do you use to conduct outreach? For example, advertising on radio or television, social media, mailings, or community outreach events. (RQ24)
What messages are you emphasizing in your recruitment efforts? For example, are you emphasizing services available or financial aid support or some other feature? (RQ24) [Site visitors: Gather outreach materials.]
What strategies have been most-effective? How do you know these are working well? From your perspective, why are these strategies effective? (RQ22)
What recruitment strategies are not working as well? How do you know these aren’t working as well? From your perspective, why are these strategies less effective? (RQ25)
Are there any challenges with recruitment? Have there been challenges garnering interest in the services or is there more interest from individuals than you can serve? If so, what do you think are the reasons? How have you overcome these challenges? (RQ25)
From your perspective, how successful have recruitment efforts been? (RQ22)
Ultimately, what have been the characteristics of America’s Promise participants? How do they differ from the target population for this effort? (RQ26)
What are participants’ demographic characteristics compared to your planned target population? (RQ26)
Are they focused on particular industries or occupations? (RQ26)
Do they require similar sets of services? (RQ17, RQ26)
To what extent do enrolled participant align with your target population? Are you able to successfully enroll the target population? (RQ23, RQ25, RQ26)
Based on your grant application and our telephone discussions, the structure and key components of the [PROGRAM NAME] services include [ADD DETAIL]. Identify key service offerings (education/training, case management, job placement, supportive services) based on grantee proposal and quarterly narrative reports prior to the visit. (RQ15)
Is this correct? What changes, if any, have been made to your service structure or model since the proposal? (RQ15)
Are all components of [PROGRAM NAME] services available throughout the region? Or is there variation in what is available based on a participant or partner’s location? Please describe. (RQ5, RQ16)
Is there a core set of services that all participants receive (such as an employment plan or set of assessments)? Do participants typically receive multiple services? If multiple services, which services are commonly combined? If yes, why do you use specific combinations of services? (RQ17)
What is the process for determining the services and supports that each participant is offered? (RQ17, RQ18)
IV. Alternative services, Outcomes, and Sustainability
A. Alternative services
Are any of these other programs similar to [PROGRAM NAME]? If so, how are they similar and how are they different? (RQ2)
Please describe any current programs or services (RQ2)
How long have these programs been in place? (RQ2)
Which organizations run these programs or services? (RQ2, RQ5)
What is the funding source for these other programs? Is it public or private funding? (RQ4, RQ5)
What is the target population for these programs? (RQ2, RQ23, RQ24)
Among [PROGRAM NAME] participants, about what percentage of participants complete the program and exit services? How much of a challenge was retaining participants? What was your targeted completion rate? What was your actual completion rate (RQ19, RQ25)
Among those who do not complete services, what are the primary reasons that participants drop out of services and/or do not complete training? What barriers do participants face in remaining engaged in services? Does the program use any incentives and/or supportive services to keep them engaged? (RQ19, RQ25)
What percentage of participants find employment? How often is that job related to the industry or occupation for which they received services and training? Are these usually full-time or part-time? What are typical salaries? (RQ20)
What barriers do participants face in finding jobs? What strategies do you use to help them overcome those barriers? What strategies appear most successful? (RQ20)
C. Plans for continued partnerships and promising practices
Which partnerships do you think are/would be most likely to last after the grant funding ends? What partnerships are likely to end? Why? (RQ10, RQ14)
What steps are you taking now to ensure sustainability of the partnerships? (RQ14)
What changes to partnerships, if any, must be made to sustain the effort? (RQ11, RQ14)
What methods of communicating or coordinating across partners are likely to be sustained? (RQ13)
Do you plan to add any partners? If so, which partners and why? (RQ14)
What are the key challenges you anticipate in sustaining partnerships? (RQ14)
How have service delivery systems in the region been affected by the [PROGRAM NAME] program? What changes, if any, have you noted in workforce development, higher education, or economic development systems? (RQ2, RQ3, RQ6)
How satisfied have you been with the [PROGRAM NAME] effort in general? What do you perceive as the most successful aspects of service provision? What are the least successful? What improvements or changes to the services would you like to see? (RQ16)
How successful has the [PROGRAM NAME] partnership been, from your perspective? Why do you think it has/has not been successful? (RQ13)
Have these partnerships influenced the region’s continuous improvement efforts in serving individuals and employers? If so, in what ways?
What challenges, if any, have you encountered in working with participants? What steps have you or the [PROGRAM NAME] partners taken to address these challenges? (RQ21, RQ25)
What promising practices for delivering sector-based career pathways services and training have been developed under the grant? (RQ4, RQ5, RQ8)
What have been the promising practices for collaboration (for example, frequent meetings, soliciting ideas for shared vision)? (RQ13)
What have been the key challenges to collaboration (for example, time requirement, conflict between partners, data-sharing issues)? (RQ12)
What do you think will be the most lasting value to your organization from having implemented [PROGRAM NAME]? (RQ22)
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Brittany English |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |