instrument #2
MASTER TOPIC GUIDE FOR SMALL GROUP INTERVIEWS
WITH STAFF
This page has been left blank for double-sided copying.
OMB Control No:
Expiration Date:
PAF in-depth implementation study
SMALL group interviewS WITH STAFF
master protocol
Prior to the site visit, we plan to:
Review documents and other data (proposals, site planning documents, site selection documentation, design and implementation survey data, and performance measures data) to extract relevant information
Customize the staff group interview protocol to confirm information collected through document review and supplement with questions about specific staff experiences and views
Tailor the introduction to each site using the template interview introduction as a guide
ORGANIZATION: |
|
|||||
NO. OF PARTICIPANTS |
|
|||||
MATHEMATICA STAFF: |
|
|
DATE: |
|
||
NOTE TO GROUP INTERVIEW FACILITATOR:
Introductory scripts are designed to guide you as you give instructions to group interview participants.
You do not have to read scripts verbatim. Please use them as a guide of what information to cover.
Please cover this information either using the language provided here or in your own words.
However, please review the privacy information precisely.
INTRODUCTORY SCRIPT
Thank you for agreeing to meet with us. As you know, we are from Mathematica Policy Research, an independent research firm. My name is [NAME OF INTERVIEWER] and this is [NAME OF NOTETAKER]. We are part of an independent research team that is studying programs around the country that are designed to reduce or delay repeat pregnancies among teen parents. Your agency is participating in this study, administered by the Office of Adolescent Health, at the Department of Health and Human Services. The purpose of our discussion today is to learn more about your experiences working with teen parents at [AGENCY] or with [NAME OF CONTROL PROGRAM]. Our discussion should last about 90 minutes.
Statement on Privacy: Your point of view is extremely valuable. We will take notes on our discussion today, so that we can accurately represent your views. We will keep your responses private. Information from this conversation will be combined with information from other discussions with similar individuals in other organizations implementing this program. Your responses will be kept private and the notes from this discussion will not be shared with anyone beyond the research team.
We will report most information we collect in aggregate. We may use quotations to illustrate findings, but in these instances, we will not report any information that will allow a quotation to be associated with you individually.
We would like to record this session. We will be using the recordings to capture illustrative quotes to use in writing study reports. We will use first names only during our conversation today, and in reporting quotes we will change the first names. Please do not provide your full name during our conversation today. We will never identify who made any statements we include in our reports. Do we have your permission to record the session?
[WAIT FOR RESPONSE]
Please keep in mind:
There are no right or wrong answers to the questions we will discuss today. Our questions are designed to encourage you to talk about your experiences with [NAME OF PROGRAM].
Please remember to speak only one at a time so that everyone’s comments can be heard.
Please try not to interrupt one another and allow each person to finish speaking before you do.
You may withdraw your participation at any point without penalty.
Please do not share what we discuss today outside of this room. Please respect one another’s privacy.
Your participation in this group discussion is entirely voluntary and you do not have to answer any questions that you do not want to answer.
Would anyone like to add any additional ground rules for our discussion?
PROJECT CONTACT INFORMATION
If you have further questions about this project or if you have a research-related problem, you may contact the Project Director, Dr. Susan Zief, by contacting Mathematica Policy Research at (609)275-2291 or szief@mathematica-mpr.com
PRE-DISCUSSION QUESTIONS
I want to emphasize again that there are no right or wrong answers to these questions. By voluntarily agreeing to participate in this study, we ask you to answer these questions with responses that are true for you.
Does everyone understand the purpose of our conversation today?
Does anyone have any additional questions before we begin?
BACKGROUND/ICEBREAKER
Many of you may already know each other, but we would also like to get to know you. Let’s begin by going around the room and giving your first name only. You do not have to provide your name if you do not want to.
What is your current role in [NAME OF PROGRAM]?
Before [NAME OF PROGRAM], had you ever worked with or participated in a teen pregnancy prevention program or another similar program?
If so, what was that program? Where was it offered?
Which communities or locations do you currently serve?
Do you live in one of the communities where your clients live?
Implementation Context
How would you characterize attitudes in your community toward efforts to prevent or delay repeat pregnancies?
Would you say the community is supportive of these efforts? Or is there considerable resistance to programs of this sort?
Prompts: among community leaders, school leaders, teachers, parents, and students, the public in general?
Please describe any previous initiatives to help and support teen parents in your target populations.
Were these initiatives successful?
Why or why not?
Are any of these initiatives still operating?
Are you aware of any other current programs for teen parents in your community?
If so, what are they?
What populations do they serve?
Intervention
How would you describe the central goal of [NAME OF PROGRAM]? What is it primarily trying to achieve?
How would you describe [NAME OF PROGRAM] to a youth who is new to your school/community?
What are main services and core components of the program?
What are some of its key features and requirements?
What methods does the program use?
Who is the program designed to help and how?
What youth outcomes is [NAME OF PROGRAM] targeting for change?
In other words, if this program is successful and does what it is supposed to do, what outcomes would you expect to observe in youth who participated?
How do you think the key features of this program will lead to these desired outcomes for participating youth?
What are the main messages you are trying to deliver to youth in the program?
Did you plan for and make any changes to [NAME OF PROGRAM] to fit your context/local needs or the needs of the youth you are targeting?
If so, in what areas are these changes? Why did you make them? Who decided to make the changes?
Prompts: changes to the curriculum; the schedule, duration, or frequency of sessions; recommended program staffing; supplemental materials used; anything dropped or omitted from the curriculum; added information or materials to address specific attitudes, needs, or challenges; other changes to respond to needs of target population
Have you had to make any changes to [NAME OF PROGRAM] after you started implementing it because of challenges or problems that came up?
Probe: Were there activities or components that were too hard to do, just did not work as expected, or required resources you did not have?
How well do you think the changes are working?
What strategies did you plan and use to identify and/or recruit eligible youth?
Probes:
Did you encounter any challenges in recruiting youth?
How did you address these challenges?
What strategies were most and least successful?
What do you think attracts youth to [NAME OF PROGRAM]? What are their main motivations for participating?
implementATION
What is your overall impression of [NAME OF PROGRAM]? Do you think it will be/is/was helpful to participants? If so, how? If not, why not?
What kinds of collaboration, if any, do you do with other staff in your organization to implement [NAME OF PROGRAM]?
Have you worked together to develop or implement planning, specific materials, activities, etc.?
With whom have you collaborated?
How did you find [NAME OF PROGRAM] compared to other programs for youth you have implemented?
Do you think it fits well with the work of AGENCY? If so, how? If not, why not?
[Probe on above question using staff survey responses]
How well have your efforts to link to and coordinate with other community service providers worked?
Probe: Have you encountered any challenges? How have you addressed these challenges?
Has [NAME OF PROGRAM] received the support/resources that you expected from program partners?
If so, what resources were these?
If not, why not?
Did you participate in training on [NAME OF PROGRAM]? If so, did you find the training useful? Why or why not?
Have you found that you have been able to use the information from the training in implementing [NAME OF PROGRAM]? Why or why not?
What kinds of additional support would you like from management and supervisory staff to implement [NAME OF PROGRAM]? How well do AGENCY policies fit with [NAME OF PROGRAM]? Do you find that they limit or support [NAME OF PROGRAM] implementation? If so, how?
Do you feel you have adequate resources in place to implement [NAME OF PROGRAM]? If not, what other kinds of resources do you need?
What is your workload at AGENCY like?
Have you found that your workload affects implementation of [NAME OF PROGRAM]? If so, how?
We are interested in the ongoing feedback you receive in implementing [NAME OF PROGRAM]. Do you receive that support and/or feedback from a supervisor or someone else in your organization? Who provides it? What do they do?
Probes:
How often does your supervisor meet with you?
Do they meet with you one-on-one? In a group with other facilitators? If in a group, how many people are involved?
Does your supervisor provide any support related to any specific topics or activities that are part of [NAME OF PROGRAM]?
Other ways in which you receive support to implement this program?
Do you receive specific feedback from your supervisor on your work as a facilitator of the program?
What kinds of program data do you or your supervisors collect as part of the monitoring process?
How is this data used? Does your supervisor include any specific program data in providing feedback or for offering guidance to improve quality of services? (For example, attendance rates, youth satisfaction surveys, quality observations, etc)
Do you receive this kind of support from someone outside your organization? Who provides it? What do they do?
Are there additional ways in which you would like your supervisor or manager(s) to support you in implementing this program? If so, what are they?
When you first started, did you receive systematic/formal training on this program? [ASK FOR A SHOW OF HANDS]
If so, please describe.
Did your training include any discussion of fidelity, that is, how to teach the curriculum as written, following the instructional practices provided during the training? (Ask in Wave I only)
At the end of your training, were you required to undergo any kind of certification process? [ASK FOR A SHOW OF HANDS]
If so, please describe.
After you completed the training, did you feel ready to deliver the program? If not, why not? What additional information or help would have helped you feel more ready to deliver the program? Do you have any unmet training needs at this time?
What types of technical assistance on [NAME OF PROGRAM] are available to you and/or other staff? Who provides it? Is this useful? Would you like more or different help of this type?
Perceived Usefulness of Program
In your experience implementing [NAME OF PROGRAM], what have you found to be its strongest or most useful components or features? What have you found to be its weakest features?
How is it helpful for (addressing the needs of) the youth you work with?
How is it useful to or helpful for staff implementing it?
How does it help or address the needs of the local communities?
Now I would like to ask some questions about the materials you used to deliver [NAME OF PROGRAM]. Overall, how useful are the program materials?
Probes:
Were the materials difficult or easy to use?
Were you comfortable using the program materials? Did anything make you uncomfortable?
Have you adapted or supplemented them? If yes, please provide examples.
How did the youth respond to the materials? Did they find them useful? Engaging?
Is there a way that the program materials could be improved to be more useful to you? Is there a way they could be improved to be more useful to participating youth?
Now I would like to ask some questions about the activities you conducted as part of [NAME OF PROGRAM]. Overall, how useful were these activities?
Probes:
Were you comfortable conducting the activities? Did anything make you uncomfortable?
Have you adapted or supplemented them? If yes, please provide examples.
How did the youth respond? Did they find them useful? Engaging?
Is there a way that the activities could be improved to be more useful to you? Is there a way they could be improved to be more useful to participating youth?
Now I want you to think about [NAME OF PROGRAM] as a whole. If you were going to change something about [NAME OF PROGRAM], what would you change, and why?
Participation and Engagement in Program Activities
In general, how engaged were participants in program activities? Did teens find the topics and content useful and engaging?
Please describe the activities which participants had concerns about or found challenging.
Was keeping appointments and attending sessions challenging for participants and for you?
How would you describe your relationship with the participants with whom you worked? How did you develop trust and rapport with them? What challenges did you face in doing so?
What do you think are the main barriers or challenges for participants who had to drop out?
What steps did you or your organization take to retain teens and overcome these barriers?
[For programs being delivered in multiple sites/locations] What were the rates of participation and retention among the different subsites?
PARTICIPANT CHARACTERISTICS
How would you describe the teen women [and their families] you are working with?
Are there specific differences between the youth you targeted and those who you enrolled?
How would you characterize their motivations in enrolling for this program? Are there differences in the motivations among youth in the different subsites?
What do you think participants want to achieve as a result of participating in the program?
How would you describe the main challenges youth you work with face (probe for examples from the different subsites, if applicable)?
What steps have they taken to address these challenges? What kinds of support do they most need?
How would you describe the typical families, home environments, and personal lives (probe for examples from each of the subsites, where applicable)? Who do the teens turn to for support and assistance?
How would you describe the different educational and/or professional backgrounds, interests, and goals (probe for examples from each of the subsites, where applicable)?
How would you describe the range of parenting goals? What types of parenting issues, problems or challenges do the teens you work with face?
How would you describe their relationships with their partners and/or spouses?
What different cultural or ethnic backgrounds do the youth you work with come from (in the different locations or subsites, where applicable)?
Do the cultural and ethnic backgrounds of the youth affect how you deliver service to the youth? If so, how?
What strengths [protective factors] do the youth you work with have (probe for differences among the subsites, if applicable)?
Prompts: parental support, delay of initiating of sexual intercourse, contraceptive use, knowledge of HIV/AIDS and STIs, low rates of teen pregnancy, good communication skills, absence of substance use, etc
Have you adapted the program in any way because of the ethnic or cultural backgrounds, experiences, needs, or specific challenges the youth that you serve face?
If so, how? What additions or adaptations did you make?
OVERALL EXPERIENCE
Finally, we would like to learn more about the overall experience of implementing this program and early lessons learned.
What aspects of [PROGRAM] have been the most difficult to implement, and why? What issues make it hard to deliver the program as intended?
What steps did you or your organization take to address these challenges? How have they worked?
What accomplishments are you most proud of, and why?
What lessons have you learned so far in implementing the program?
What aspects of the program do you think are most effective?
If you could change the program, what would you change (frequency, setting, staffing, topics, materials, etc)?
Do you think that the study sites will continue to implement [PROGRAM] in the future? What factors do you think will support or facilitate the sustainability of [PROGRAM] within your organization or other partner organizations? What factors do you think will make sustainability more difficult?
Closing/Thank you
Is there anything else anyone would like to say about [NAME OF PROGRAM] before we wrap up?
Thank you for participating in this discussion!
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0990-xxxx . The time required to complete this information collection is estimated to average one hour per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | SAsheer |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |