|
|
Initial Site Visit Protocol – Grantee and Partner Administrative Staff |
INITIAL (YEAR 1) INTERVIEW PROTCOL
Grantee and Partner Administrative Staff Interview
The purpose of the interview is to obtain information from Grantee Administrative Staff and Partners about their involvement in developing and implementing the tribal HPOG program. The following topics are addressed:
A. Background 2
C. Tribal HPOG Program Structures 3
4. Program Design and Curriculum 5
5. Qualifications of Implementation Staff 5
6. Skills and Competencies for Local Health Care Workforce 6
2. Incorporation of Family Education Model 7
4. Implementation Facilitators and Challenges 7
1. Participant Educational Attainment 8
2. Participant Employment Outcomes 8
3. Role of tribal HPOG program in building tribal health workforce capacity 9
4. Satisfaction with tribal HPOG program 9
F. Recommendations for Program Improvement 9
G. Conclusion 9
All background information relevant to these topics will be consulted prior to the interview in order to provide contextual information.
Tribal HPOG program |
|
Study ID
|
|
Interviewed by
|
|
Date & time |
|
This interview is being conducted for the Evaluation of the Tribal Health Professions Opportunity Grants. The Health Profession Opportunity Grants (HPOG) program is administered by the Administration for Children and Families (ACF), an agency within the U.S. Department of Health and Human Services. ACF funded 32 five-year demonstration projects to design and implement innovative health workforce development training programs targeting Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals. Five of the 32 demonstration projects were awarded to Tribal Organizations and Tribal Colleges to develop culturally-informed training programs. The Tribal HPOG Evaluation is a comprehensive evaluation of the design, implementation, and outcomes of the five Tribal HPOG programs. The interview questions will focus on your perceptions of the Tribal HPOG program, including the program design and curriculum, recruitment, supportive services, family engagement, the quality of instruction, educational attainment and employment outcomes of participants, implementation barriers and facilitators, and overall satisfaction. Your participation is voluntary, but it is very important because your responses will help us to improve the program. As explained in the consent form you signed, we will keep information about you private and you will not be identified in any report or publication of this study or its results. You may decline to answer any question you wish. If you have any questions, please let me know.
NOTE: This interview protocol will be tailored based on the specific role of an individual. All sections may not be applicable to every individual. It is unlikely that any one individual will be asked all questions in this protocol.
What is the name of your agency or organization?
How long has it been in existence? What is its history?
What are the range of services and programs provided?
What is the agency’s or organization’s service area?
What is your role in the organization/agency?
What are your role and responsibilities for the Tribal HPOG project?
Planning
[Partners only] How was the agency invited to participate in the tribal HPOG program by [insert lead grantee organization]?
Tell me about your decision to apply for the Tribal HPOG Program. What did you expect or hope for? (e.g., opportunity to work with partners, interest in health professions focus of the program)?
Were you involved in the planning for the program? YES NO
If no, skip to the next section.
If yes, continue with the questions below.
Who else was involved in the planning process? How were different viewpoints incorporated into program planning?
What issues were addressed? Were there challenges that required a compromise?
Tribal HPOG Program Structures
Have grantees incorporated structures necessary to enhance the health care workforce needs of the community?
[Ask Partner Administrative staff as appropriate]
Program Type
[READ ALOUD TO RESPONDENTS BEFORE ASKING 12a-12e]We are interested in learning about the program type (e.g., academic instruction, on the job training, apprenticeship) and why this type is being used.
Please identify type of program model:
Academic instruction + occupational skills training
OJT + training
Apprenticeship
Other
Why was this model chosen?
What are the key program components?
Remediation (e.g., basic/foundational skills)
Occupational skills training (e.g., resume building)
Was the program incorporated within, or as an extension of, an existing program? Please describe.
[If no]: What was in place prior to the [insert program name]?
What, if any, resources have been leveraged to support this program?
[READ ALOUD TO RESPONDENTS BEFORE ASKING 13a-13b] Next we would like to learn about the organizational structure of the program. ?
[Lead organization only] How is the program administered (provide sample org. chart?)
[Lead organization only] What strategies are in place within the administrative structure to support student recruitment and retention?
[READ ALOUD TO RESPONDENTS BEFORE ASKING 14a-14b]We are now interested in discussing the partnerships, both new and existing, that have been formed to support the project and the strategies used to engage these partners in the project.
[Lead organization only a - g]
[Lead organization only] What partnerships have been formed to deliver training or program services? Specifically, what is the role of public human service agencies (e.g., TANF, housing, substance abuse, disability and other agencies), the public workforce investment system (e.g., Workforce Investment Act programs), the criminal justice system (e.g., corrections, parole/probation, juvenile justice), employer and employment agencies, educational institutions, faith-based and community-initiatives, and other service providers?
Which partners are tribal vs. non-tribal?
Describe your partnerships with employers? Formal or informal?
[Lead organization only] Have you worked with any of the project partners before?
[Lead organization only] Describe recruitment and outreach to project partners, including potential education institutions and employers.
[Lead organization only] What strategies are used for collaboration and coordination across all project partners? How are they established (formal and informal)?
[Lead organization only] How is the partnership functioning? Are partners meeting their obligations per the MOU? Have you run into any challenges? If yes, please describe.
[Lead organization only] On a scale of 1 to 5, where 1 is ‘not at all likely’ and 5 is ‘very likely’. how likely are you to work with these partners again?
[Lead organization only] What community engagement strategies are used to inform project planning and implementation (e.g. advisory boards, council of elders, open community meetings/ gatherings)?
[Partners only h-i]
[Partner only] Describe your relationship with the [insert lead organization].
Have you worked with this organization before?
How is the partnership functioning? Have you run into any challenges? If yes, please describe. On a scale of 1 to 5, where 1 is ‘not at all likely’ and 5 is ‘very likely’, how likely are you to work with this organization again?
[Partner only] Aside from [insert lead organization] do you work closely with any of the other Tribal HPOG project partners? If yes:
Have you worked with this organization before?
How is the partnership functioning? Have you run into any challenges? If yes, please describe.
On a scale of 1 to 5, where 1 is ‘not at all likely’ and 5 is ‘very likely’, how likely are you to work with this organization again?
[READ ALOUD TO RESPONDENTS BEFORE ASKING 15a-15g]We are now interested in learning more about the program curriculum, such as academic lectures and use of field practicum training manuals, and in learning about ways in which the program was designed or modified for Tribal populations?
Describe the career pathways offered by your program.
Describe the licensure/certification process for each career pathway, as applicable.
How portable is the licensure/certificate (i.e., can it be transferred over state lines)?
Describe the elements of the training program and its curriculum/curricula. If details made available in other program materials, then skip specifics listed below.
Competencies to be developed (e.g., skill requirements of the target occupation)
Pre-requisites
Duration of training program
Who provides training
Program location
Convenience
Accessibility
Why was this curriculum/model/approach chosen? Were other curricula/models/approaches considered? If so, which ones?
Did the evidence for this curriculum/model/approach show relative advantage over other programs?
Were adaptations or modifications made to the training program based on local conditions or preferences? Based on partner input?
Was the model/curriculum adapted to be culturally relevant? If yes, then what adaptation strategies are being used to deliver a culturally relevant curriculum?
Did the tribe request a waiver for any required program elements per the FOA? (e.g., apprenticeship, other)
Qualifications of Implementation Staff
[READ ALOUD TO RESPONDENT BEFORE ASKING 16a-16b]. Next we would like to learn about the qualifications of the program implementation staff.
How were staff responsible for implementing the program curriculum recruited and/or selected? What staff qualifications were considered? What about previous experience working with tribal populations?
Did you face any challenges in staff recruitment?
Skills and Competencies for Local Health Care Workforce
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 17a-17c] We would like to learn about how the training program addresses skills and competencies needed by the local and/or regional health care industry.
What community characteristics shape participants’ employment opportunities? (To include general labor market conditions in the area, the extent and nature of job opportunities, and industry skill initiatives).
Were tribal and/or local (off reservation; surrounding area) workforce needs assessed? If so, how?
Were other sources of information/data were used to determine the fit between the training program and local industry needs? If so, what?
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 18a-18b] Now we would like to learn about how the social, economic, and political context of the community influence program design and implementation.
Describe barriers to education/training and employment for population served.
For example, socio-cultural barriers include language and communication differences; practices that differ from their own beliefs and traditions; fear and mistrust of [TBD] institutions, and a lack of knowledge about how to navigate the system. Barriers include childcare, housing, transportation, health, mental health, substance abuse, domestic violence, GED, training, adult basic education, English language learning/literacy.
What community factors influence the design and implementation of the program (e.g., ongoing recession, organizational priorities, social tensions)? Please describe.
Have grantees implemented processes to prepare participants for employment in the tribal health care sector?
[Ask Partner Administrative staff as appropriate]
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 19a-19e] Next we would like to ask you some questions about the support services that are offered as part of the program and how they are incorporated.
What types of services or incentives are being provided to participants?
Social services (e.g., food stamps, childcare)
Employability services (e.g., essential skills, life skills, job readiness)
Employment related services (e.g., job development and placement, job coaching)
Program retention services (e.g., mentoring)
Job retention services (e.g., mentoring, peer support groups)
Are supportive services culturally-based? If so, please describe.
How are service components sequenced and coordinated? How are they designed to address participants’ needs and unique barriers to employment?
Describe any challenges experienced in providing supportive services. What strategies were used to overcome the challenge(s) noted?
Which supportive services, if any, have been most effective at enabling students to participate and complete the program?
Incorporation of Family Education Model
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 20a-20b] We would like to discuss any strategies that are used to engage the families of program participants.
What strategies are used to engage families to support the participant?
Participation in orientation
Participation in college activities
Provide support for studying
Other
What is the rationale for engaging families?
Use Program Data
[lead organization only] [READ TO THE RESPONDENT BEFORE ASKING QUESTIONS 21a-21c] Now we would like to ask a few questions about the collection and use of program data.
Is there a process in place for entering data into the Performance Reporting System?
How are data collected from partners?
Who is responsible for entering data the Performance Reporting System?
Are data used for program management decisions, performance monitoring, or program correction? Do you share these data with partners? Stakeholders? Participants? Examples might be dashboard indicators, interim milestones reports, reports on outcomes sorted by cohorts, individual participant reports.
Services received
Attendance
Placement results
Employment
Program drop-outs
Terminations
Retention follow-up information
Supportive services information
Are there any challenges to collecting and using the program data? If so, please explain.
Implementation Facilitators and Challenges
[READ TO THE RESPONDENT BEFORE ASKING QUESTIONS 22a-22e] Next we would like to ask you some questions that will help us to understand whether or not the program was implemented as intended and what barriers you faced in implementing the program.
Have you experienced any start-up challenges? If yes, please describe.
Have you experienced any implementation challenges? If yes, please describe.
Describe the training for staff responsible for implementing the program curriculum. How were they oriented to the program?
Describe the process for supervision and communication with program implementation staff? Are these staff mentored?
Has there been any implementation staff turnover? If yes, how was this handled? Do you think this had an effect on program implementation?
Is there evidence that participants in the program achieved successful employment and work force capacity building outcomes?
[Ask Partner Administrative staff as appropriate]
[Participant Educational Attainment – Role of Tribal HPOG program in building Tribal Health Workforce Capacity --- ONLY ask if there are program completers at the time of site visit]
Participant Educational Attainment
[READ ALOUD TO THE RESPONDENT BEFORE ASKING QUESTIONS 23a-23d] We would like to learn more about the educational attainment of program completers, and if participating in the program resulted in a degree/certificate, and/or leads to the successful receipt of a license or certification issued through a state or industry entity
How do you define program completion? (e.g., receive degree)
In general, have you found that program completers have moved on to a professional or industry recognized certificate or licensure?
What program components (e.g., competency based curricula, supportive and cultural services, family education/engagement, employment and employability related activities) are important to program participation and completion?
Have you found that any of the program components are not as useful in supporting program participation and completion as anticipated?
Participant Employment Outcomes
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 24a-24d] Now we would like to ask questions about the employment outcomes of participants. ?
What are participants’ employment outcomes?
Employed full-time
Employed part-time
Serving internship
Unemployed
What types of jobs do participants have?
With what employers? Note: Need to note tribal hiring preferences & policies for employment (tribal member, spouse of tribal member, member of another tribe)
Tribe
In area
Out of area
Other tribe
Non-tribal
IHS
Urban Indian Clinics
Local health care
Out of area
Role of tribal HPOG program in building tribal health workforce capacity
[READ ALOUD TO RESPONDENTS BEFORE ASKING QUESTIONS 25a-25b] Building on the last section, we would like to ask a few questions about whether the program has been able to help fill vacancies in the local or regional tribal health workforce.
Are participants serving their own community or another Tribal community? Provide examples.
Did participants encounter any barriers in finding employment in a Tribal community? If so, what were these barriers?
Satisfaction with tribal HPOG program
[READ ALOUD TO THE RESPONDENT BEFORE ASKING QUESTIONS 26a-26c] Now we would like to hear you perception of the satisfaction level of program stakeholders.
On a scale of 1 to 5, where 1 is “not satisfied at all” and 5 is “very satisfied”, how satisfied are you with the program?
To your knowledge, are
participants satisfied with the program?
partners satisfied with the program? [Ask partners directly if they are satisfied]
employers satisfied with the program?
In your opinion, what have been some of the key benefits for participants served through the tribal HPOG program?
Recommendations for Program Improvement
Is there anything that you would change about the program that could be helpful to future participants?
Is there anything you would like to add before concluding the interview?
Thank you very much for your time. It has been a pleasure to speak with you.
|
|
Follow-up Site Visit Protocol – Grantee and Partner Administrative Staff |
FOLLOW UP 1 AND 2 INTERVIEW PROTCOL
Grantee and Partner Administrative Staff Interview
The purpose of the interview is to obtain information from Grantee Administrative Staff and Partners about their involvement in developing and implementing the tribal HPOG program. The following topics are addressed:
B. Tribal HPOG Program Structures 11
2. Program Design and Curriculum 12
3. Skills and Competencies for Local Health Care Workforce 12
2. Incorporation of Family Education Model 13
4. Implementation Facilitators and Challenges 13
1. Participant Educational Attainment 14
2. Participant Employment Outcomes 14
3. Role of tribal HPOG program in building tribal health workforce capacity 14
4. Satisfaction with tribal HPOG program 15
E. Recommendations for Program Improvement 15
F. Conclusion 15
All background information relevant to these topics will be consulted prior to the interview in order to provide contextual information.
Tribal HPOG program |
|
Study ID
|
|
Interviewed by
|
|
Date & time |
|
This interview is being conducted for the Evaluation of the Tribal Health Professions Opportunity Grants. The Health Profession Opportunity Grants (HPOG) program is administered by the Administration for Children and Families (ACF), an agency within the U.S. Department of Health and Human Services. ACF funded 32 five-year demonstration projects to design and implement innovative health workforce development training programs targeting Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals. Five of the 32 demonstration projects were awarded to Tribal Organizations and Tribal Colleges to develop culturally-informed training programs. The Tribal HPOG Evaluation is a comprehensive evaluation of the design, implementation, and outcomes of the five Tribal HPOG programs. The interview questions will focus on your perceptions of the Tribal HPOG program, including the program design and curriculum, recruitment, supportive services, family engagement, the quality of instruction, educational attainment and employment outcomes of participants, implementation barriers and facilitators, and overall satisfaction. Your participation is voluntary, but it is very important because your responses will help us to improve the program. As explained in the consent form you signed, we will keep information about you private and you will not be identified in any report or publication of this study or its results. You may decline to answer any question you wish. If you have any questions, please let me know.
NOTE: This interview protocol will be tailored based on the specific role of an individual. All sections may not be applicable to every individual. It is unlikely that any one individual will be asked all questions in this protocol.
Background [Only if not previously interviewed]
What is the name of your agency or organization?
How long has it been in existence? What is its history?
What are the range of services and programs provided?
What is the agency or organization’s service area?
What is your role in the organization? For the Tribal HPOG project?
[Partners only] How was the agency invited to participate in the tribal HPOG program by [insert lead organization]?
Were you involved in the planning for the program? If yes, describe.
Tribal HPOG Program Structures
Have grantees incorporated structures necessary to enhance the health care workforce needs of the community?
[READ ALOUD TO RESPONDENTS BEFORE ASKING 8a-8f]We are now interested in discussing the partnerships, both new and existing, that have been formed to support the project and the strategies used to engage these partners in the project.
Are you aware of any changes to the program partnership since the last time we spoke (or beginning of program if new respondent)? If yes, please describe.
[Lead organization only] How is the partnership functioning?
Are partners meeting their obligations per the MOU? Have you encountered any challenges? If yes, please describe.
Have you run into any challenges? If yes, please describe.
[Partners only] Describe your relationship with [insert prime organization] and any other project partners you work closely with.
How is the partnership functioning?
Have you run into any challenges? If yes, please describe.
On a scale of 1 to 5, where 1 is ‘not at all likely’ and 5 is ‘very likely’, how likely are you to work with these partners again?
[Lead organization only] What strategies have been used for collaboration and coordination across project partners? How are they established? Which, if any, are most effective?
[Lead organization only] What community engagement strategies are used to inform project planning and implementation (e.g. advisory boards, council of elders, open community meetings/ gatherings)? Which, if any, are most effective?
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 9a-9c] We are now interested in learning more about the program curriculum, such as academic lectures, use of field practicum training manuals, and in learning about ways in which the program was designed or modified for Tribal populations.
Are you aware of any changes to the program curriculum since the last time we spoke (or beginning of program if new respondent)? If yes, please describe. Why was the curriculum changed?
Are there aspects of the program curriculum that you believe are more effective with regard to program participation and completion than others? Please describe.
Are you aware of any problems/challenges with the program curriculum? If yes, please describe.
Skills and Competencies for Local Health Care Workforce
[READ ALOUD TO RESPONDET BEFORE ASKING QUESTION 10a] We would like to learn about how the training program addresses skills and competencies needed by the local and/or regional health care industry.
What community characteristics shape participants’ employment opportunities? (To include general labor market conditions in the area, the extent and nature of job opportunities, and industry skill initiatives).
[READ ALOUD TO THE RESPONDENT BEFORE ASKING QUESTIONS 11a-11b] Next we would to learn about how the social, economic, and political context of the community influenced program design and implementation.
Describe barriers to education/training and employment for population served. For example, Socio-cultural barriers include language and communication differences; practices that differ from their own beliefs and traditions; fear and mistrust of [TBD] institutions, and a lack of knowledge about how to navigate the system. Barriers include childcare, housing, transportation, health, mental health, substance abuse, domestic violence, GED, training, adult basic education, English language learning/literacy.
What community factors influence the implementation of the program (e.g., ongoing recession, organization priorities, social tensions)? Please describe.
Have grantees implemented processes that successfully prepare participants for employment in the Tribal health care sector?
[Ask Partner Administrative staff as appropriate]
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 12a-12d] Now we would like to ask you some questions about the support services that are offered with the program and how they are incorporated.
Are you aware of any changes to the types of supportive services offered through the program since the last time we spoke (or beginning of program if new respondent)? If yes, please describe.
Social services (e.g., food stamps, childcare)
Employability services (e.g., essential skills, life skills, job readiness)
Employment related services (e.g., job development and placement, job coaching)
Program retention services (e.g., financial)
Job retention services (e.g., mentoring, peer support groups)
Have students utilized these services?
Describe any challenges experienced in providing supportive services. What strategies were used to overcome the challenge(s) noted?
Which supportive services, if any, have been most effective at enabling students to participate in and complete the program?
Incorporation of Family Education Model
[If family engagement model part of program] [READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 13a-13b] We would like to discuss any strategies that are used to engage the families of program participants.
Are you aware of any changes to strategies used to engage families to support the participant?
What strategies, if any, have facilitated participant participation and completion in the program?
Program Data
[lead organization only] [READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 14a-14c] Now we would like to ask a few questions about the collection and use of program data.
Have you used any program data for program management decisions, performance monitoring, or program correction? If yes, describe how these data are used?
Do you share these data with partners? Stakeholders? Participants?
Are there any challenges to collecting and using the program data? If so, please explain.
Implementation Facilitators and Challenges
[READ ALOUD BEFORE ASKING QUESTIONS 15a-15b]Next we would like to ask you some questions that will help us to understand whether or not the program was implemented as intended and what barriers you faced in implementing the program.?
Have you experienced any implementation challenges? If yes, please describe.
Have you experienced any challenges supervising and communicating with program implementation staff? Are these staff mentored?
Has there been any implementation staff turnover? If yes, how was this handled? Do you think this had an effect on program implementation?
Is there evidence that participation in the program resulted in successful employment and work force capacity building outcomes?
[Ask Partner Administrative staff as appropriate]
Participant Educational Attainment
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 16a-16e] We would like to learn more about the educational attainment of program completers, and if participating in the program results in a degree/certificate, and/or leads to successful receipt of a licensure or certification issues through a state or industry entity.
In general, have you found that program completers have moved on to a professional or industry recognized certificate or licensure?
Were there other indicators of success identified and achieved (e.g., increased life skills, self-efficacy, confidence, social supports)?
Do you think participants achieved self-sufficiency (e.g., number who no longer receive public assistance)? Provide examples.
What program components (e.g., competency based curricula, supportive and cultural services, family education/engagement, employment and employability related activities) are important to program participation and completion?
Have you found that any of the program components are not as useful in supporting program participation and completion as anticipated?
Participant Employment Outcomes
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 17a-17f] Now we would like to ask questions about the employment outcomes of participants.
What are participants’ employment outcomes?
Employed full-time
Employed part-time
Serving internship
Unemployed
What types of jobs do participants have? Provide examples.
With what employers? Note: Need to note tribal hiring preferences & policies for employment (tribal member, spouse of tribal member, member of another tribe)
Tribe
In area
Out of area
Other tribe
Non-tribal
IHS
Urban Indian Clinic
Local health care
Out of area
Are you aware of whether participants have retained their jobs? For how long? Were job retention services provided?
Are you aware of how participant earnings compare to pre-participation earnings?
Do you know if any have experienced some type of employment advancement? Position? Higher pay? More hours? Responsibilities?
Role of tribal HPOG program in building tribal health workforce capacity
[READ ALOUD TO RESPONDENT BEFORE ASKING QUESTIONS 18a-18b] Building on the last section, we would like to ask a few questions about whether the program has been able to help fill vacancies in the Tribal health workforce.
Are participants serving their own community or another Tribal community? Provide examples. If students not serving Tribal populations, why not?
Did participants encounter any barriers in finding employment in a Tribal community? If so, what were these barriers?
Satisfaction with tribal HPOG program
[READ ALOUD TO RESPONENT BEFORE ASKING QUESTIONS 19a-19c] Now we would like to hear you perception of the satisfaction level of program stakeholders.
On a scale of 1 to 5, where 1 is ‘not satisfied at all’ and 5 is ‘very satisfied’, how satisfied are you with the program?
To your knowledge, are
participants satisfied with the program?
partners satisfied with the program? [Ask partners directly if they are satisfied]
employers satisfied with the program?
In your opinion, what have been some of the key benefits for participants served through the tribal HPOG program?
Recommendations for Program Improvement
Is there anything that you would change about the program that could be helpful to future participants?
Is there anything you would like to add before concluding the interview?
Thank you very much for your time. It has been a pleasure to speak with you.
File Type | application/msword |
Author | Heather Langerman |
Last Modified By | Department of Health and Human Services |
File Modified | 2011-10-17 |
File Created | 2011-10-17 |