Feedback After the Office of Planning, Research, and Evaluation's 2024 Research and Evaluation Conference on Self-Sufficiency (RECS)

Instrument II. Post-RECS Data Collection Focus Group Facilitation Guide

While participants are joining the call and waiting for the focus group to start, the notetaker can share a link to the short background information survey [content on last page of this document] and ask participants to complete the survey.

Introduction and Consent (5 minutes)

Good morning/afternoon! Thank you for joining us.

My name is [Interviewer/Facilitator Name]. I am joined here by my colleague(s) [Name]. We work for a not-for-profit research organization called the American Institutes for Research, also known as AIR. The Administration for Children and Families’ (ACF) Office of Planning, Research, and Evaluation (OPRE) is partnering with AIR to gain a deeper understanding of the utility of the content shared during the Research and Evaluation Conference on Self-Sufficiency (RECS) 2024. We invited you to speak with us because you attended RECS in May this year and you expressed interest in participating in a focus group. Today we will be asking you to share information and feedback on your goals for attending RECS, learning experience at the conference, and whether and how you have been able to use information you learned at the conference.

We anticipate that this discussion will take up to 90 minutes. Please know that we will not share your responses directly with OPRE staff or mention you by name in any of our reports. I want to assure you that all information obtained today will be kept private to the extent allowable by law and will be used only for the purposes of this study. It is completely up to you whether you’d like to chat with us today or answer particular questions. As a thank you for your participation, we will send you a $100 honorarium after today’s discussion.

Next, I’m going to share my screen with a copy of the informed consent form. This document will provide an overview of the purpose of this data collection as well as your rights as a participant in this interview.

[facilitator to share screen with a copy of the informed consent form]

Please take some time to review the informed consent document and let us know if you have any questions.

[Pause for 2-3 minutes to allow the respondent to read document]

Do you have any questions about this?

[answer questions that may arise]

With that, we would like to check if each participant here consents to participating in this focus group. If you do, please say “yes”, or enter “Yes” in the chat.

[wait for each participant to offer written (via chat) or verbal consent. If a participant does not offer consent, let them know that you understand and ask them to leave the call before proceeding]

Finally, I would like to record the conversation to make sure the information we gather is accurate. It will also allow us to focus on our discussion and limit the distraction of note taking. We save the recordings on a secure server that only AIR team members can access. All recordings will be destroyed after the completion of this study. Are you OK with us recording this discussion?

[If YES] Thank you. If at any point during our conversation you wish for me to turn off the recording, please let me know, and I will do so. Do you have any questions before we begin?

[If NO - take detailed notes as best you can.] No problem. Do you have any questions before we begin?

Goals & Needs (10-15 minutes)

First, we’d like to learn a bit about you, your current roles, and work context.

1. Can you please briefly introduce yourself by sharing your name, role, employer, and how long you have been in your role?

2. Based on the needs of your job/role, which topics do you need to stay up to date on?

   1. Thank you for sharing the main topics you are interested in. Next, we are going to ask you to answer a quick polling question so we can capture everybody’s responses. The poll question is similar to the question you just responded to about the topics you need to stay to date on, and the response options in this poll are the RECS tracks.

      1. Anonymous Poll on main topics of interest: Based on the needs of your job and organization, which topics do you need to stay up to date on?

         1. TANF Programs, Policies, and Populations

         2. Employment and Mobility in the Labor Market

         3. Youth Well-Being and the Transition to Adulthood

         4. Strengthening Families, Fatherhood, Marriages, and Relationships

         5. Evaluating Social Programs, Building Evidence, and Using Data

         6. Approaches to Alleviate Poverty and Expand Opportunity

      2. [Briefly reflect on poll findings and ask if anyone would like to share additional thoughts.] 

3. What were you hoping to gain by attending RECS this year?

   1. [Alternate wording] What were your goals when you decided to go to RECS this year?

   2. If the attendees need some probing/examples of goals: Networking, professional development (research and/or policy-related), learn about work to implement in my own work, sharing research and/or policy effort, speaking at the conference.

Learning Experiences at RECS Conference (25 – 30 minutes)

Next, we’d like to dive into some questions on how you used the information you learned at RECS this year.

4. Were there specific issues that you or your organization were hoping to gain insight into while at RECS?

   1. If yes, what were these issues? Probe on broad issues: Self-sufficiency research, welfare research, and/or current federal policy efforts

   2. To what extent did you gain the insight you needed from what you learned at RECS?

5. While you attended RECS this year, what process, tools, and/or resources did you use to capture the information that was shared at the conference?

   1. Probes: Written notes, typed notes, printed materials shared by conference organizers, notes in the program book offered at RECS, note taking feature on conference app offered at RECS.

   2. Which of these processes, tools, and/or resources did you find most helpful and why?

   3. If you didn’t use any of these processes, tools, and/or resources, why not? Did you use any other approaches to capture the information shared at RECS 2024?

6. Since the conference, how have you accessed the information you learned at RECS?

   1. Probes: Written notes, typed notes, printed materials shared by conference organizers, notes in the program book offered at RECS, notes emailed to self from the conference app offered at RECS.

   2. If participants have not accessed the information they captured at RECS: Could you tell us why you may not have accessed the information you captured?

      1. Probe: was there a barrier to accessing the information; have you not needed to reference back to the information yet?

Use of Conference Research and Information (30 – 35 minutes)

Next, we’d like to understand how you might have used the information you learned at RECS. The use of this information might look like different things for different people. For example, it could be that the information you learned might allow you to think about an issue differently or more expansively, it could inform decision making, you might share the information to strengthen a relationship, or use it to bolster an argument for a previous decision.

7. Could you share with us a few examples of things you learned at RECS?

8. Can you think of how something you learned at RECS changed your understanding of a particular issue(s)?

   1. Probes: Examples of how what you learned changed your understanding of an issue could look different for different people. For example:

• Changed the way you look at problems facing your organization or the people your organization serves?

• Expanded your understanding of an issue?

• Suggested alternative solutions to a problem encountered by either your organization or by the people your organization serves?

• Brought attention to an issue your organization or the people your organization serves have not yet faced?

• Provided a framework for structuring improvement efforts (e.g., for a program, research study, policy implementation, evaluation)?

• Provided a common language and set of ideas for you and your colleagues?

• Helped you to develop or strengthen a professional relationship?

2. Increased your ability to share knowledge with others on an issue(s)?Probes: What was the issue? What was the information that changed your understanding of this issue? How frequently has the information you learned at RECS changed your understanding of a particular issue(s)?

9. Can you think of an example of how something you learned at RECS has informed a decision you had to make at work?

1. Probes: Examples of how what you learned could impact decision-making could look like informing: research or evaluation projects; the development or implementation of a new program or policy; implementation of particular interventions or programs with a specific population; scaling up a program; redesign or elimination of an existing program or policy; directing new or additional resources (funds or people) to a particular program or research priority

2. Probes: What was the context? What was the information that informed your decision making? Has this information impacted decision-making in more than one context?

10. Aside from RECS, what are your trusted sources of information on research on human services programs and policies?

    1. What contributes to these sources of information being trustworthy and useful in your work?

    2. If participants share literal sources (like email newsletters, social media, conferences) Who publishes these sources (e.g., names of organizations or people)?

    3. If they respond with people/organizations: How do you learn about the research from these people or entities (e.g., newsletters, social media, convenings)?

11. What changes would you suggest to RECS to make it easier to retain and use the information presented?

Wrap up (4-5 minutes)

12. Is there anything else you would like to share with us about your RECS learning experience and use of the information you learned at RECS that we haven’t covered?

Conclusion (1 minute)

Thank you again for taking the time to speak with us today! Your feedback will help OPRE improve future conference attendees’ learning experiences and help ensure RECS is informative and useful to your work moving forward. If you have not already completed your demographic questionnaire, we ask that you take the time to complete it prior to leaving this focus group.

Thank you once again for your time!

RECS 2024 Focus Group: Participant Information

Thank you for participating in our focus group today. To help us better understand who is in attendance, please provide the following information. We will get started shortly.

Background and Demographic Information

1. Please start by providing initials below. 
   For example, "John Smith" would enter "JS"

___________________________________________

2. Was RECS 2024 your first-time attending RECS? (select one)

• Yes

• No

3. How many sessions did you participate in RECS 2024?

• One session

• Less than half of the sessions

• Half the sessions

• More than half of the sessions

• Most/all sessions

4. Which of the following best represents how you attended RECS 2024? (select one):

• In-person only

• Mostly in-person and some virtual

• About half in-person and half virtual

• Mostly virtual and some in-person

• Virtual only

5. What is your total number of years of work experience (in any professional field or role)?

• Less than 1 year of work experience

• 1-2 years

• 3-5 years

• 6-10 years

• 11-20 years

6. What is your highest level of education completed?

• Some high school

• High school diploma/GED

• Some college

• Associate’s degree

• Bachelor’s degree

• Master’s degree

• Doctoral degree

• Other: [please specify]

• Prefer not to say