Interview Protocols for MH Portfolio Awardees OMB Number 0906-0059, Expiration Date XX/XX/202X
Attachment B2. Maternal Health Portfolio Evaluation Interview Protocol for Awardees (Awardee Leadership, Evaluators, and Activity Implementation Staff)
Public Burden Statement: This information is collected as part of a portfolio-wide evaluation of Maternal Health (MH) programs funded by the Health Resources and Services Administration. The evaluation will help the HRSA Maternal and Child Health Bureau identify individual and/or collective strategies, interrelated activities, and common themes within and across the MH programs that may be contributing to or driving improvements in key maternal health outcomes. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0906-0059 and it is valid until XX/XX/202X. This information collection is voluntary. Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Information Collection Clearance Officer, 5600 Fishers Lane, Room 14N39, Rockville, Maryland, 20857 or paperwork@hrsa.gov.
The Evaluator will tailor each protocol to the awardee’s program prior to conducting the interview. Evaluators will determine, in collaboration with HRSA, which activities the awardee is implementing based on program documents. The activities will all correspond with data collection elements of the seven MH Portfolio strategies from the evaluation design, including: 1) Establishing and Strengthening Partnerships and State Capacity; 2) Workforce Training and Expanding the Maternal Health Workforce; 3) Providing Technical Assistance; 4) Improving Access and Coordinating Care; 5) Implementing Quality Improvement Initiatives, including Maternal Safety Bundles; 6) Implementing Telehealth; and 7) Creating and Disseminating Products.
|
Introductory Script and Informed Consent (Verbal)
Good morning/afternoon. My name is [NAME] and I am a [TITLE] at [ORGANIZATION]. the HRSA Maternal Health Portfolio Evaluation, an evaluation of five HRSA programs including [AWARDEE’S GRANT PROGRAM].
If speaking with awardee respondent: Is this still a good time to conduct the interview?
If no: Can we schedule another time to conduct the interview that works for you?
If yes: Before we get started, I am going to read some information about the evaluation and your participation in the interview. It will only take a few minutes.
[ORGANIZATION] is conducting an evaluation for the Health Resources and Services Administration, or HRSA, about addressing maternal mortality. We are interested in speaking with you today to learn about barriers and facilitators to implementing your program, how your program is addressing health equity, as well as contextual factors and potential indicators for scaling and replicating your program’s strategies and activities.
This interview is being conducted for the evaluation of HRSA’s Maternal Health Portfolio Project. The portfolio includes five maternal health programs, including the [insert name of program] program. We are interested in speaking with you today to learn about your program’s progress toward achieving your goals over the past year, barriers and facilitators to implementing your program, experiences receiving TA from MHLIC and/or AIM, how your program is addressing health equity, as well as contextual factors and potential indicators for scaling and replicating your program’s strategies and activities. As a point of clarification, although we know you do a wide range of work focused on maternal health, for the purposes of this conversation (and the portfolio eval overall), we would like to focus only on those programs funded by HRSA for your [insert name of program, e.g., State MHI program].
Our discussion today will last about one hour. Information from this interview will be included in documents associated with the Maternal Health Portfolio evaluation for HRSA. While we will not use your name or any others in these reports, it may be possible to identify you through your position or through other details that you share in this interview. The information we gather will help HRSA to understand contextual factors and potential indicators for scaling and replicating your programs strategies and activities.
If you have questions about the study, please contact the Maternal Health Data Lead, Dianna Frick, dfrick@hrsa.gov.
Do you have any questions about the evaluation or your participation in the interview today?
Do you agree to participate in this interview?
If "Yes" then proceed.
If No: Thank you for your time. [Stop the interview].
My colleague [NAME] is also on the call and will take notes during our conversation. We will also create an audio recording of the interview. We will use the recording to create a transcript to inform our report. The transcript will not include your name or contact information. We will delete the recording at the end of the project.
Do you agree to have this interview recorded?
If respondent says "yes" then proceed. BEGIN RECORDING.
If "no" then say: "That's fine. You may still participate in the interview. Please be patient as I take notes." DO NOT BEGIN RECORDING.
I’d first like to start off by asking you a few questions about your program.
Based on your application, the goals of the project are [list project goals]. Have any changes been made to the stated goals? If yes, please describe.
What is/are the target population(s) of your program? Prompts if respondent is unsure – ask for each program component:
Racial/ethnic group
Geographic area
Socio-economic status
Have there been any contextual factors, like policies or other state-level maternal mortality efforts or programs that have played a role in the implementation of the program? Prompts if respondent is unsure:
Pre-existing partnerships
State and national health policies
Maternal Mortality Review Committees (MMRC)
Title V
Number of hospital closures and OB closures state-wide
Who are the main partner organizations involved in program implementation? For each partner organization please describe their role.
Based on program documents, including your grant application and your program progress report, your program is working on the following activities [insert activities]. Is this correct? Have you made any changes?
We would like to know more about how your program is implementing these activities. In the next part of the interview, I am going to ask you a set of questions about each activity.
For your program efforts related to [ACTIVITY #1]:
What are the goals for this activity?
What are the main objectives associated with implementing this activity?
(Follow-up questions for activities related to Providing Technical Assistance only) How does the technical assistance you provide address health equity? Telehealth? Maternal safety bundles?
The following set of questions will ONLY be asked for the following activities:
|
For the activity, please describe the implementation settings.
Prompt if respondent is unsure: Hospital, Community health center, Behavioral health center, Non-medical setting
Who is/are the target population(s) of the activity?
Prompt if respondent is unsure: Race/ethnicity, Geography, Socio-economic status, Education
Please describe the staff involved in implementing the activity.
Prompt if respondent is unsure: Administrative staff, Physicians, Medical Assistants, Health department staff, Policy makers, Implementing organization vs partner organization.
Next, we are interested in the potential to replicate or scale this aspect of your program.
What are the unique or important characteristics of this activity, including approaches that are evidence-based, that would be essential to someone trying to replicate the program?
What resources, such as infrastructure, cost, space, materials, and technical capacity, have been used for implementing this activity? Please indicate whether the resource was from your organization or a partner organization.
What contextual factors have played a role in the implementation of this activity?
Prompt if respondent is unsure: Pre-existing partnerships, State and national health policies, Maternal Mortality Review Committees (MMRC), Title V, Number of hospital closures and OB closures state-wide.
The Evaluator will repeat Section 1 of questions for each program activity.
|
I’d like learn a bit more about how your program is collecting, managing, and reporting data for program evaluation.
Could you please describe the current status of your evaluation design, data collection, data analysis, and data reporting?
What challenges have you encountered with data collection and how have you addressed them?
To what degree are your data collection activities being implemented on the intended time frame? If not on time, why not?
Now I’d like to find out about the barriers and facilitators you’ve encountered during the planning and start-up phases of the program.
To what degree is the program being implemented as intended? If there have been any changes, please describe.
Did you or the program as a whole experience any challenges during the planning and start-up phases of the program? If yes, please describe. Prompts if the respondent is unsure:
State or local policies, partnerships, or existing maternal health programs/initiatives (e.g., Title V, MMRC, Perinatal Quality Collaborative, Levels of Care Designations),
Leadership buy-in (e.g., awardee-level or activity-level),
Staffing (e.g., adequate number of staff, turnover, appropriately trained staff),
Access to required infrastructure,
Access to additional resources and support,
Technical assistance,
Target population factors (e.g., social determinants of health such as poverty rates, social norms, health literacy, transportation access, unemployment)
Factors outside of program control (e.g., contextual factors, national or regional events, COVID-19)
What did you or the program as a whole find to be helpful during the planning and start-up phases of the program?
Now I’d like to find out about how your program is addressing maternal health equity.
How does your program ensure that it meets the cultural and linguistic needs of your target population(s)?
CLAS Reference for Interviewer to provide as prompts if the respondent is unsure:
Principal Standard
Provide effective, equitable, understandable, and respectful quality care and services that are responsive to diverse cultural health beliefs and practices, preferred languages, health literacy, and other communication needs.
Governance, Leadership and Workforce
Advance and sustain organizational governance and leadership that promotes CLAS and health equity through policy, practices, and allocated resources.
Recruit, promote, and support a culturally and linguistically diverse governance, leadership, and workforce that are responsive to the population in the service area.
Educate and train governance, leadership, and workforce in culturally and linguistically appropriate policies and practices on an ongoing basis.
Communication and Language Assistance
Offer language assistance to individuals who have limited English proficiency and/or other communication needs, at no cost to them, to facilitate timely access to all health care and services.
Inform all individuals of the availability of language assistance services clearly and in their preferred language, verbally and in writing.
Ensure the competence of individuals providing language assistance, recognizing that the use of untrained individuals and/or minors as interpreters should be avoided.
Provide easy-to-understand print and multimedia materials and signage in the languages commonly used by the populations in the service area.
Engagement, Continuous Improvement, and Accountability
Establish culturally and linguistically appropriate goals, policies, and management accountability, and infuse them throughout the organization's planning and operations.
Conduct ongoing assessments of the organization's CLAS-related activities and integrate CLAS-related measures into measurement and continuous quality improvement activities.
Collect and maintain accurate and reliable demographic data to monitor and evaluate the impact of CLAS on health equity and outcomes and to inform service delivery.
Conduct regular assessments of community health assets and needs and use the results to plan and implement services that respond to the cultural and linguistic diversity of populations in the service area.
Partner with the community to design, implement, and evaluate policies, practices, and services to ensure cultural and linguistic appropriateness.
Create conflict and grievance resolution processes that are culturally and linguistically appropriate to identify, prevent, and resolve conflicts or complaints.
Communicate the organization's progress in implementing and sustaining CLAS to all stakeholders, constituents, and the general public.
How is your program addressing bias among professionals and healthcare providers? If yes, how?
How is your program reducing barriers to accessing care among target populations? If yes, how? Prompts if the respondent is unsure: Race/ethnicity, Geography, Socio-economic status, Education
How is your program implementing quality improvement initiatives to improve quality of care among target populations?
Prompts if the respondent is unsure: Race/ethnicity, Geography, Socio-economic status, Education
I’d first like to start off by asking you about updates and program progress over the past year.
Have there been any changes to your program plan over the past year? If yes, please describe any changes to the program – including specific activities – over the past year. Probes/re-worded question if the respondent is unsure: To what degree is the program being implemented as intended? If there have been any changes, please describe
Please describe the program’s progress towards meeting each goal.
Last year we discussed the activities that your program is implementing. Based on program documents, including your grant application and your program progress report, and what you just described, your program is working on the following activities [insert activities]. Is this right? Have there been any changes or activities I missed?
I would like to know more about how your program has implemented these activities over the past year. In the next part of the interview, I am going to ask you a set of questions about each activity.
For your program efforts related to [ACTIVITY #1]:
What were the main objectives associated with implementing this activity in the past year?
(Follow-up questions for Providing Technical Assistance only) How does the technical assistance you provide address health equity? Telehealth? Maternal safety bundles?
The following set of questions will ONLY be asked for the following activities:
|
For the activity, please describe the implementation settings.
Prompt if respondent is unsure: Hospital, Community health center, Behavioral health center, Non-medical setting
Who is/are the target population(s) of the activity?
Prompt if respondent is unsure: Race/ethnicity, Geography, Socio-economic status, Education, Providers
Please describe the staff involved in implementing the activity, including project staff and partner staff.
Prompt if respondent is unsure: Administrative staff, Physicians, Medical Assistants, Health department staff, Policy makers, Implementing organization vs partner organization
Next, we are interested in asking about the potential to replicate or scale this aspect of your program.
What are the unique or important characteristics of this activity, including approaches that are evidence-based, that would be essential to someone trying to replicate the program?
What contextual factors have played a role in the implementation of this activity?
Prompt if respondent is unsure: Pre-existing partnerships, State and national health policies, Maternal Mortality Review Committees (MMRC), Title V, Number of hospital closures and OB closures state-wide
To what degree were changes made to the implementation of the activities based on quality improvement/evaluation efforts?
The Evaluator will repeat Section 1 of questions for each program activity.
|
I’d like learn a bit more about how your program is collecting, managing, and reporting data for program evaluation.
Could you please describe the current status of your evaluation design, data collection, data analysis, and data reporting?
What challenges have you encountered with data collection and how have you addressed them?
To what degree are your data collection activities being implemented on the intended time frame? If not on time, why not?
Now I’d like to find out about your program’s experiences receiving technical assistance.
What technical assistance did you receive from the Maternal Health Learning and Innovation Center (MHLIC) over the past year?
How would you describe the quality, knowledge, and effectiveness of the trainer?
How would you describe the quality, relevance, and timeliness of the content?
Did you have to consult additional resources to meet your technical assistance need?
To what extent did the technical assistance contribute to the program’s ability to meet program goals?
What technical assistance did you receive from AIM over the past year?
How would you describe the quality, knowledge, and effectiveness of the trainer?
How would you describe the quality, relevance, and timeliness of the content?
Did you have to consult additional resources to meet your technical assistance need?
To what extent did the technical assistance contribute to the program’s ability to meet program goals?
Now I’d like to find out about the barriers and facilitators you’ve encountered in the past year while planning and implementing the program.
Did you or the program as a whole experience any challenges during the past year? If yes, please describe.
More specifically, have you had any challenges with any of the following:
State or local policies, partnerships, or existing maternal health programs/initiatives (e.g., Title V, MMRC, Perinatal Quality Collaborative, Levels of Care Designations),
Leadership buy-in (e.g., awardee-level or activity-level),
Staffing (e.g., adequate number of staff, turnover, appropriately trained staff), Access to required infrastructure,
Access to additional resources and support,
Technical assistance (if not discussed above)
Target population factors (e.g., social determinants of health such as poverty rates, social norms, health literacy, transportation access, unemployment)
Factors outside of program control (e.g., contextual factors, national or regional events, COVID-19)
What did you or the program as a whole find to be helpful during the past year of program implementation in terms of making progress towards your project goals and objectives?
Now I’d like to find out about how your program is addressing maternal health equity.
How does your program ensure that strategies and activities are meeting the cultural and linguistic needs of your target population(s)?
CLAS Reference for Interviewer to provide as prompts if the respondent is unsure:
Principal Standard
Provide effective, equitable, understandable, and respectful quality care and services that are responsive to diverse cultural health beliefs and practices, preferred languages, health literacy, and other communication needs.
Governance, Leadership and Workforce
Advance and sustain organizational governance and leadership that promotes CLAS and health equity through policy, practices, and allocated resources.
Recruit, promote, and support a culturally and linguistically diverse governance, leadership, and workforce that are responsive to the population in the service area.
Educate and train governance, leadership, and workforce in culturally and linguistically appropriate policies and practices on an ongoing basis.
Communication and Language Assistance
Offer language assistance to individuals who have limited English proficiency and/or other communication needs, at no cost to them, to facilitate timely access to all health care and services.
Inform all individuals of the availability of language assistance services clearly and in their preferred language, verbally and in writing.
Ensure the competence of individuals providing language assistance, recognizing that the use of untrained individuals and/or minors as interpreters should be avoided.
Provide easy-to-understand print and multimedia materials and signage in the languages commonly used by the populations in the service area.
Engagement, Continuous Improvement, and Accountability
Establish culturally and linguistically appropriate goals, policies, and management accountability, and infuse them throughout the organization's planning and operations.
Conduct ongoing assessments of the organization's CLAS-related activities and integrate CLAS-related measures into measurement and continuous quality improvement activities.
Collect and maintain accurate and reliable demographic data to monitor and evaluate the impact of CLAS on health equity and outcomes and to inform service delivery.
Conduct regular assessments of community health assets and needs and use the results to plan and implement services that respond to the cultural and linguistic diversity of populations in the service area.
Partner with the community to design, implement, and evaluate policies, practices, and services to ensure cultural and linguistic appropriateness.
Create conflict and grievance resolution processes that are culturally and linguistically appropriate to identify, prevent, and resolve conflicts or complaints.
Communicate the organization's progress in implementing and sustaining CLAS to all stakeholders, constituents, and the general public.
How is your program addressing bias among professionals and healthcare providers?
How is your program reducing barriers to accessing care among target populations? Prompts if the respondent is unsure:
Race/ethnicity
Geography
Socio-economic status
Education
How is your program implementing quality improvement initiatives to improve quality of care among target populations? Prompts if the respondent is unsure:
Race/ethnicity
Geography
Socio-economic status
Education
Have there been any changes to your program plan over the past year? If yes, please describe any changes to the program – including specific activities – over the past year. Probes/re-worded question if the respondent is unsure: To what degree is the program being implemented as intended? If there have been any changes, please describe
Have there been any major changes that changed the course of your work throughout the program?
Please describe the program’s progress towards meeting each goal.
We know you have been working on a range of activities, but since we only have limited time with you today, we would like to go into depth on just a few of them. We would like to focus on activities that you find the most innovative, activities that have the potential for scale/spread, activities that are going well, and activities where you have experienced the most challenges.
Let’s start with activities that are innovative.
5. Out of all of the activities that your program is implementing, which do you feel are the most innovative? Why? Prompt if respondent is unsure: what are the goals/objectives; method or approach; target population; geographic location; implementation setting; targeted health outcome.
Next, we are interested in asking about activities that have the potential for replication or scale. Do you have any activities that could be, or are already being, scaled or replicated?
6. What are the goals/objectives of this activity? What are the unique or important characteristics of this activity, including approaches that are evidence-based, that would be essential to someone trying to replicate the program?
7. What contextual factors have played a role in the implementation of this activity over the past year?
Prompt if respondent is unsure: Pre-existing partnerships, State and national health policies, Maternal Mortality Review Committees (MMRC), Title V, Number of hospital closures and OB closures state-wide
8. Do you think this activity could be successfully scaled? Why? By scaled we mean, implemented in a larger setting or with a larger population. Prompt if respondent is unsure:
Observed outcomes
Stakeholder buy-in/satisfaction
9. What conditions would be necessary for this part of your program to be successfully scaled? Prompt if respondent is unsure:
Financial resources required
Staff roles required
Activity processes
Political will or policy support required
Infrastructure required
Resources needed by patients (e.g., transportation)
10. Could this activity be successfully implemented in other contexts? Why? Prompt if respondent is unsure:
Observed outcomes
Stakeholder buy-in/satisfaction
11. What conditions would be necessary for this part of your program to be successfully implemented in other contexts? Prompt if respondent is unsure:
Financial resources required
Staff roles required
Activity processes
Political will or policy support required
Infrastructure required
Resources needed by patients (e.g., transportation)
12. Are there parts of this activity that would be challenging or too difficult to scale or spread (implement in other contexts)? Why? Prompt if respondent is unsure:
Observed outcomes
Stakeholder buy-in/satisfaction
Financial resources required
Staff roles required
Activity processes
Political will or policy support required
Infrastructure required
Resources needed by patients (e.g., transportation)
Next, we are interested in asking about an activity that is going well and an activity that you find challenging.
13. Out of all of the activities you are implementing, what is an activity you find is GOING WELL? Please tell us more about this activity.
What are the goals/objectives for this activity?
Please describe the implementation settings. Prompt if respondent is unsure: Hospital, Community health center, Behavioral health center, Non-medical setting
Who is/are the target population(s) of the activity? Prompt if respondent is unsure: Race/ethnicity, Geography, Socio-economic status, Education
Please describe the staff involved in implementing the activity. Prompt if respondent is unsure: Administrative staff, Physicians, Medical Assistants, Health department staff, Policy makers, Implementing organization vs partner organization
Have any of these (goals/objectives, implementation setting, target population, staff involved) changed over the past year?
14. Out of all of the activities you are implementing, what is an activity you find CHALLENGING? Please tell us more about this activity.
What are the goals/objectives for this activity?
Please describe the implementation settings. Prompt if respondent is unsure: Hospital, Community health center, Behavioral health center, Non-medical setting
Who is/are the target population(s) of the activity? Prompt if respondent is unsure: Race/ethnicity, Geography, Socio-economic status, Education
Please describe the staff involved in implementing the activity. Prompt if respondent is unsure: Administrative staff, Physicians, Medical Assistants, Health department staff, Policy makers, Implementing organization vs partner organization
Have any of these (goals/objectives, implementation setting, target population, staff involved) changed over the past year?
What is challenging about this activity?
15. Which of your activities do you think are having the greatest impact on maternal health outcomes?
Now I’d like to find out about the barriers and facilitators you’ve encountered in the past year while planning and implementing the program.
15. Did you or the program as a whole experience any challenges during the past year? Over the course of the program? If yes, please describe.
More specifically, have you had any challenges with any of the following:
State or local policies, partnerships, or existing maternal health programs/initiatives (e.g., Title V, MMRC, Perinatal Quality Collaborative, Levels of Care Designations),
Leadership buy-in (e.g., awardee-level or activity-level),
Staffing (e.g., adequate number of staff, turnover, appropriately trained staff), Access to required infrastructure,
Access to additional resources and support,
Technical assistance (if not discussed above)
Target population factors (e.g., social determinants of health such as poverty rates, social norms, health literacy, transportation access, unemployment)
Factors outside of program control (e.g., contextual factors, national or regional events, COVID-19)
16. What did you or the program as a whole find to be helpful during the past year of program implementation in terms of making progress towards your project goals and objectives?
Now I’d like to find out about your program’s experiences receiving technical assistance.
17. What technical assistance did you receive from the Maternal Health Learning and Innovation Center (MHLIC) over the past year?
How would you describe the quality, knowledge, and effectiveness of the trainer?
How would you describe the quality, relevance, and timeliness of the content?
Did you have to consult additional resources to meet your technical assistance need?
To what extent did the technical assistance contribute to the program’s ability to meet program goals?
Do you have any feedback on how the TA could be more effective?
18. What technical assistance did you receive from AIM over the past year?
How would you describe the quality, knowledge, and effectiveness of the trainer?
How would you describe the quality, relevance, and timeliness of the content?
Did you have to consult additional resources to meet your technical assistance need?
To what extent did the technical assistance contribute to the program’s ability to meet program goals?
Do you have any feedback on how the TA could be more effective?
Now I’d like to find out about how your program is addressing maternal health equity.
19. How is your program addressing health equity?
Now I’d like learn a bit more about how your program is collecting, managing, and reporting data for program evaluation.
20. Could you please describe the current status of your evaluation design, data collection, data analysis, and data reporting?
21. What challenges have you encountered with data collection and how have you addressed them?
22. To what degree were changes made to the implementation of the activities based on quality improvement/evaluation efforts?
23. Do you have any preliminary findings that you can share with us?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Emily Phillips |
| File Modified | 0000-00-00 |
| File Created | 2024-07-22 |