Focus Group Protocol for Training and TA Staff - no cover sheet

Focus Group Protocol: Training and Technical Assistance Providers

Introduction and Consent Script (10 minutes)

Hello and thank you for joining this focus group today. My name is [NAME OF FACILITATOR], and I’m joined by my colleague [NAME OF CO-FACILITATOR]. We are here from the Data Governance Support Project, which is funded by the Division of Data and Improvement within the Administration for Children and Families’ Office of Planning, Research, and Evaluation. As a part of this project, we are working with the Office of Head Start to help them review and improve the quality of the data collected through the Program Information Report, which you may also know as the “PIR.” This review is being carried out by SRI International, an independent, nonprofit research institute, under a contract for the U.S. Department of Health and Human Services.

A Federal agency may not conduct or sponsor, and no individual or entity is required to respond to, nor shall an individual or entity be subject to a penalty for failure to comply with a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless that collection of information displays a currently valid OMB Control Number. The OMB # is 0970-0531 and the expiration date is 9/30/2025.

As a part of this review, we will be sharing findings from focus groups with the Office of Head Start and the Division of Data and Improvement. Any results we share will be summarized across all conversations we have with central office, regional, and local staff, and with other stakeholders. You will not be identified by name in any form of analysis or report, and data will be reported in a de-identified manner. In addition, when we need to use quotes to help illustrate the findings, we will only identify them by role of the participant (such as, “technical assistance provider”). In some cases, our team may follow up with you to request additional feedback about your comments. All data collected for the review will be stored in secure environments, and we will protect the privacy of the information you provide. Finally, your participation in this focus group is voluntary, and you may decline to respond at any time.

At this time, I am going to turn on a recording to capture consent for participation in this focus group and recording.

[Turn on recording]

We would like to record this conversation to ensure we accurately capture your comments. If you agree, we will retain the recording only until we can validate our notes, at which point we will destroy the audio recording. Since we are in a group setting today, if you are not comfortable being recorded at this time, we ask that you exit the call, and we will follow up with you at a later time. At this time, I will pause to make sure everyone we have with us agrees to allow us to record the focus group. Please do so by typing “yes” in the Zoom chat or stating “yes” verbally.

[Facilitators to make note of anyone leaving the call, confirm that all remaining have consented]

Today, our main topics for discussion will be about the PIR data, including how you use it in your job role, your understanding of the quality the PIR data, and your thoughts about ensuring that PIR data is useful for understanding Head Start Programs from a local, regional, and national perspective. The PIR is an annual data collection that provides comprehensive data on the services, staff, children, and families served by Head Start and Early Head Start programs nationwide. All grant recipients and delegates are required to submit a PIR for Head Start and Early Head Start programs. We have scheduled 90 minutes for this focus group. We would like everyone to feel comfortable sharing open feedback, so we ask that everyone here hold our conversation in private. We will be using a tool called “Mentimeter” to ask interactive, polling-style questions; responses on Mentimeter cannot be tied to you and are only used to inform our conversation. We also welcome you to use the Zoom chat to type comments in addition to speaking. Before we get started, I would like to allow you all to ask any questions you have about our work or the plan for our time together today.

[Address any comments or questions]

Questions and Prompts

Section 1: Use of Program Information Report (PIR) Data (25 minutes)

To start, we’d like to spend a few minutes getting to know you all and learning about your familiarity with the PIR data and how you use it in your work.

Topic	TTA Provider Questions & Prompts
Introductions	Briefly, can you tell us your name and a little bit about your role as a training and technical assistance provider? [Facilitator to call name of individuals from the list]
Assessing familiarity with PIR data	We’d like to use a poll to quickly get some information about how frequently you use PIR data. We have shared a link to a website called Mentimeter in the chat. You can open this link; please keep it open on your desktop as we’ll be returning to it a few more times. [Mentimeter multiple choice style question:] How frequently do you use PIR data as a part of your job duties? Less than once a year About once a year A few times a year About once a month About twice a month More than twice a month
Understanding how staff use PIR data	How do you use PIR data in your role? Prompt [provide descriptors of some potential data uses if group cannot identify]: Using data could look a lot of different ways. It could be reviewing data to get an idea of the “state” of programs, looking at changes across multiple years, reviewing specific items to inform work you are doing, looking at a particular program’s data to understand aspects of the program including about the services they provide, or having discussions with a program that involve PIR data.
Understanding how staff use PIR data	When you review and use PIR data, what sections or topics are you most likely to look at and why?

Section 2: Perceptions of Data Quality (40 minutes)

As a part of this project work, we reviewed the last few years of PIR data in detail, as well as reviewing questions submitted by programs to the Head Start Enterprise System help desk and questions posed by programs about the PIR on the online MyPeers community, to gain an understanding of program experiences when submitting the PIR each year. We learned that some program staff can find it difficult to collect and maintain the large variety of data collected for the PIR, and that it can be challenging to translate data that’s used in day-to-day program administration into one report that summarizes the program’s entire year.

We’d like to spend our next few questions digging into the quality of PIR data as you interact with it. Just a note: When we refer to “data quality,” we mean PIR data that is submitted in a timely manner, is complete, and accurately represents all the important work done by a program.

Topic	TTA Provider Questions & Prompts
Understanding support/technical assistance provided around PIR data	How do you receive or provide support to colleagues, OHS staff, or local programs around PIR data?
Understanding support/technical assistance provided around PIR data	What feedback do you hear from colleagues, OHS staff, or local programs about the PIR? Prompt [if staff report that they do not hear feedback about the PIR]: PIR data aside, what common questions or issues do programs approach you about that might in any way be related to data collected on the PIR?
Identifying mismatches between expected and reported data	Thinking back to instances you have used PIR data, were you surprised by anything you saw in the data?
Gauging perception of data quality	Now we would like to do a quick polling activity in Mentimeter. [Facilitator provides directions about advancing to next question.] The link will display a list of topics that are collected on the PIR. We’d like you to reflect on the extent to which you are confident that the data is an accurate reflection of the services provided by programs. The intent is to understand, for each of these topics, if there are any data quality concerns based on your perceptions. [Mentimeter “slider” style question; left-hand side indicating low level of confidence in data, right-hand side indicating high level of confidence in data] Topics: Enrollment and eligibility data Child transition and turnover data Workforce or staffing data (e.g., qualifications, turnover, salaries) Health status data (e.g., insurance, immunization, BMI, EPSDT, oral health, chronic conditions) Family composition, education, and employment data Family services data IDEA and disabilities data Coaching data Collaboration agreements data [Identify three topics with lowest level of confidence] What are some signs you see that indicate there may be data quality issues with [topic 1]? What are some signs you see that indicate there may be data quality issues with [topic 2]? What are some signs you see that indicate there may be data quality issues with [topic 3]? Are there any topics about the PIR that you would like to call out, but that were not included on the list?
	What are your general perceptions of the quality of PIR data? Prompt: Are there specific PIR topics that you feel are sometimes inaccurate? What gives you this perception?

Section 3: Looking Forward and Wrap-Up (15 minutes)

We have 15 minutes remaining together. As we wrap up, I’d like to ask some final questions about how to make the PIR data as useful as possible and leave some time for any of your final thoughts.

Topic	TTA Provider Questions & Prompts
Making PIR data useful	In a perfect world, what practices could the Office of Head Start implement to help programs collect, maintain, and submit high-quality PIR data? Prompt: This could be changing the submission schedule of the PIR, changing the reporting structure of PIR to focus on centers or individuals, or the creation of new practices that help programs collect and submit high-quality data [Co-facilitator to enter these into Mentimeter “rankings” question as identified] Which of these practices would you prioritize as most important to improve the quality and usefulness of PIR data?
Making PIR data useful	What could OHS do to help make the PIR data more accessible and easier to use?
Additional thoughts	Before we wrap up today, we want to leave a few minutes for anyone to share any thoughts about the PIR data that you have not yet had a chance to say. What final reflections do you have about the PIR?

Those are all questions we have for today. We want to thank you all for taking the time to participate. We appreciate your insights and comments.

1