Strengthening the Implementation of Marriage and Relationship Programs (SIMR)

SIMR Staff Interview Topic Guide OMB Control Number: 0970-0531

Expiration Date: 7/31/2022

staff interview Topic Guide for simr

Note to reviewers: This instrument includes a universe of questions relevant to a broad range of strategies that will be tested in the Strengthening the Implementation of Marriage and Relationship Education (SIMR) project. The instrument will be tailored and shortened for each individual site and the strategy that it is testing.

INTERVIEWER NOTE: Based on your current understanding of the program and previous conversations with the program, use this topic guide to identify relevant topics that then will be tailored to the program. To do this, first, identify topics where we have not already collected information. Then, use the identified topics to develop program-specific questions.

Not all programs will have staff in the roles identified in the table. To map staff roles in the table to specific programs, use the following definitions:

• Program leaders and managers: Individuals responsible for the overall direction and management of the program with a high-level understanding of the program’s mission.

• Program supervisors: Those who oversee program implementation, provide support to the frontline staff, and provide information to program leaders and managers.

• Frontline staff: Staff responsible for serving parents, children, and families enrolled in the program; this can include frontline staff at a partner agency, if appropriate. Separate interviews will be conducted for staff (or small groups of staff) who work with different populations in the program.

Introduction and consent

Thank you for taking the time to speak with us today. We are from Mathematica, an independent research firm, and we are here to learn about your experiences using [strategy]. My name is [NAME] and my colleague is [NAME].

We are speaking today on behalf of the Strengthening the Implementation of Marriage and Relationship Programs project, which we call “SIMR” for short. SIMR is a study sponsored by the Administration for Children and Families within the U.S. Department of Health and Human Services. Through this project, [program] has been working with Mathematica and our partner, Public Strategies, to design and test strategies to address common implementation challenges.

We will be speaking today about your program’s use of [strategy]. We will ask you some questions regarding the strategy including training or materials you received, your use and comfort with the strategy, participant’s responses, and suggested improvements. Before we start, I want to let you know that your participation in this interview is voluntary, and you may stop at any time. Do you consent to participate in this interview?

Providing information is voluntary, and all individual responses that are collected will be kept private to the extent permitted by law. We expect this discussion to take about 45 minutes. There are no right or wrong answers. We value the information you will share with us and want to make sure we capture it all by recording it. If you do not agree to the recording, you can still participate, and we will not record it, but we have someone who will take notes. Only the team that is working on the study will have access to them. We will destroy the recording and the transcription at the end of the study.

Do we have your permission to record the discussion?

Now, I am going to read a statement:

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-0531, Expiration: 07/31/2022. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Robert Wood; RWood@Mathematica-mpr.com.

Do you have any questions before we get started?

NOTE: The Paperwork Reduction Act Statement: This collection of information is voluntary and will be used to gather information for the purpose of rapid-cycle learning activities to strengthen programs. Public reporting burden for this collection of information is estimated to average 45 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-0531, Exp: 07/31/2022. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Robert Wood; RWood@Mathematica-mpr.com

Topics to cover in interviews

Topic	Program leaders and managers	Program supervisors	Frontline staff
Feedback on training, guidance, or materials
Usefulness and clarity of training, guidance, or materials received as a part of SIMR			
Change(s) in knowledge and behaviors from receiving training, guidance, or materials			
Confidence in skills gained from receiving training, guidance, or materials			
Overall satisfaction with training, guidance, or materials			
Suggested changes and/or additions to training, guidance, or materials			
Use of strategy
Fidelity of implementation (i.e., did you use each part of the strategy, as developed?) (for program leaders and managers and supervisors: observations of fidelity)			
Frequency of strategy use			
Reasons for not using the strategy, if strategy was not implemented			
Barriers and facilitators to implementation of strategy			
Comfort with strategy (for supervisors: perceived comfort)			
Extent to which strategy has made their job easier or improved their ability to complete their job successfully			
Participant responsiveness
Observed participant response to strategy (e.g., improved relationships between participants and with staff, increased attendance, improved engagement, etc.)			
Observed changes in participant skills/behaviors			
Extent to which strategy affects recruitment, retention, or engagement
Perceptions of whether it improves service delivery, addresses participant needs, or achieves intended outcomes			
Perceptions of whether strategy effectiveness differs across circumstances or populations			
Perception or observed changes of strategy’s effect on recruitment, enrollment, or engagement			
Data collection
How the program assesses fidelity and role of staff in different positions			
Additional data collected outside of nFORM (e.g., incentives, attendance, recruitment) and how it’s used to identify areas for improvement	
Frequency that program shares and discusses data with partners	
Frequency of internal staff conversations around data (e.g., program leaders/managers discuss with supervisors, supervisors discuss with frontline staff)			
Challenges collecting, entering, and/or processing data to assess success of strategy			
Additional data program staff would like to collect or analyze to assess success of strategy			
Suggested improvement overall
Parts of strategy that worked well and why			
Parts of strategy that didn’t work well and why			
Desired or recommended adaptions to the strategy and why this change may improve the strategy			
Desired or recommended changes to program policies/procedures to better accommodate the strategy and why that may improve implementation of the strategy			
If partner organizations are involved in implementing the strategy
Frequency of communication with partner agency			
Process of communication with partner agency			
Nature and focus of communication related to the strategy			
Satisfaction with partner agency regarding strategy or change			

DRAFT 3