Office of Head Start Program Information Report Data Quality Focus Groups

Focus Group Protocol: Local Staff

Introduction and Consent Script (10 minutes)

Hello and thank you for joining this focus group today. My name is [NAME OF FACILITATOR], and I’m joined by my colleague [NAME OF CO-FACILITATOR]. We are working with the Office of Head Start to help them review and improve the quality of the data collected through the Program Information Report, which you may also know as the “PIR.” We appreciate your willingness to share your experiences and perspectives related to preparation, submission, and use of the PIR data. We are conducting this review under SRI International’s contract with the Division of Data and Improvement in the Office of Planning, Research, and Evaluation of the Administration for Children and Families.

A Federal agency may not conduct or sponsor, and no individual or entity is required to respond to, nor shall an individual or entity be subject to a penalty for failure to comply with a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless that collection of information displays a currently valid OMB Control Number. The OMB # is 0970-0531 and the expiration date is 9/30/2025.

As a part of this review, we will be sharing findings from focus groups with the Office of Head Start and the Division of Data and Improvement. Any results we share will be summarized across all conversations we have with local, regional, and central office staff, and with other stakeholders. You will not be identified by name in any form of analysis or report, and data will be reported in a de-identified manner. In addition, when we need to use quotes to help illustrate the findings, we will only identify them by role of the participant (such as, “program director” or “data manager”). In some cases, our team may follow up with you to request additional feedback about your comments. Except for what is already public, all data collected for the review will be stored in secure environments, and we will protect the privacy of the information you provide. Finally, your participation in this focus group is voluntary, and you may decline to respond at any time.

At this time, I am going to turn on a recording to capture consent for participation in this focus group and recording.

[Turn on recording]

We would like to record this conversation to ensure we accurately capture your comments. If you agree, we will retain the recording only until we can validate our notes, at which point we will destroy the audio recording. The transcripts will be destroyed at the end of the project with the Office of Head Start. Since we are in a group setting today, if you are not comfortable being recorded at this time, we ask that you exit the call, and we will follow up with you at a later time about providing comments during in a separate, unrecorded session. At this time, I will pause to make sure everyone we have with us agrees to allow us to record the focus group. Please do so by typing “yes” in the Zoom chat or stating “yes” verbally.

[Facilitators to make note of anyone leaving the call, confirm that all remaining have consented]

Today, our main topics for discussion will be about the PIR data, including how your program collects and maintains data that is ultimately reported on the PIR, your understanding of the quality of the PIR data, and your thoughts about how the Office of Head Start can help programs report high-quality PIR data. We have scheduled 90 minutes for this focus group. We would like everyone to feel comfortable sharing open feedback, so we ask that everyone here hold our conversation in private. We will be using a tool called “Mentimeter” to ask interactive, polling-style questions; responses on Mentimeter cannot be tied to you. We also welcome you to use the Zoom chat to type comments in addition to speaking. Before we get started, I would like to allow you all to ask any questions you have about our work or the plan for our time together today.

[Address any comments or questions]

Questions and Prompts

Section 1: Use of Program Information Report (PIR) Data (20 minutes)

To start, we’d like to spend a few minutes getting to know you all and learning about your familiarity with the PIR data and how you use it in your work. Before we start, we’d like to go around and have everyone tell us their name and a little bit about their role at their program.

Topic	Questions and Prompts	Questions to skip if not enough time
Assessing familiarity with PIR data	We’d like to use a poll to quickly get some information about how frequently you interact with PIR data. We have shared a link to a website called Mentimeter in the chat. You can open this link; please keep it open on your desktop as we’ll be returning to it a few more times. [Mentimeter multiple choice question:] How frequently do you collect, maintain, or submit PIR data, or support other staff with data entry, as a part of your job duties? Less than once a year About once a year A few times a year About once a month More than once a month
Assessing PIR data responsibilities	Now, we’d like to ask another polling question to understand more about how you help with PIR data. [Mentimeter select all that apply question:] Which of the following descriptions apply to you? Please mark all that apply: I manage staff who input enrollment data when families enroll in our program. I input enrollment data when families enroll in our program. I manage staff who enter other data throughout the year (such as services data or staff data) that is used to populate the PIR I enter data throughout the year that is used to populate the PIR I am primarily responsible for checking data that is used to populate the PIR (e.g., reviewing it throughout the year to make sure it is complete and accurate) I assist in checking data that is used to populate the PIR I am primarily responsible for submitting the PIR report in the Head Start Enterprise System I assist in submitting the PIR report in the Head Start Enterprise System I am primarily responsible for reporting program data (including PIR data) for internal or external uses (such as to the governing body) I assist in reporting program data (including PIR data) for internal or external uses [Other]

Section 2: Perceptions of Data Quality (45 minutes)

As a part of this project work, we reviewed the last few years of PIR data, as well as questions programs submitted to the Head Start Enterprise System help desk and questions programs posted about the PIR on the online MyPeers community, to understand program experiences with submitting the PIR each year. We learned that some program staff encounter difficulties when they collect and maintain the large variety of data collected for the PIR, and that it can be challenging to translate data that’s used in day-to-day program administration into one report that summarizes the program’s entire year.

We’d like to spend our next few questions digging into the quality of PIR data as you understand it. Just a note: When we refer to “data quality,” we mean PIR data that is submitted in a timely manner, is complete, and accurately represents all the important work done by a program.

Topic	Central Office Questions and Prompts	Questions to skip if not enough time
Understanding challenges related to collecting, maintaining, and submitting data	Our next few questions are about three specific areas on the PIR. Based on our review of the PIR data, help desk questions, and MyPeers posts these are areas where programs noted that they needed additional guidance to ensure their data matches what’s asked for on the PIR. For each of these topic areas, we’d like to ask for your input about what, if any, challenges your program experiences in collecting, maintaining, or submitting high-quality data. These could be challenges that arise when multiple people enter data, challenges with data that you must frequently check or update during the year, issues getting your data management software to correctly “count” or aggregate data, challenges related to a lack of guidance from OHS about how to “count” or track data, or others. The first area we want to explore is data about the children and pregnant women served by their programs. For example, some programs asked for help with tracking and reporting primary eligibility data. What, if any, challenges does your program experience related to collecting, maintaining, and submitting: Demographic and eligibility data? Child transition and turnover data? Prompt: How did you address these challenges? Prompt: Were there any challenges you were not able to address?
Understanding challenges related to collecting, maintaining, and submitting data	The next area we want to discuss is the child and family services data. We chose this because many programs reported that they provided specific family services to no families, while others reported that they provided specific family services to all families. Few programs reported that they provided specific family services to some of their families (that is, some families received a service, some did not). What, if any, challenges does your program experience related to collecting, maintaining and submitting: Health status data (e.g., insurance, immunization, BMI, EPSDT, oral health, chronic conditions)? Family composition, education, and employment data? Family services data? Individuals with Disabilities Education Act (IDEA) and disabilities data? Prompt: How did you address these challenges? Prompt: Were there any challenges you were not able to address?
Understanding challenges related to collecting, maintaining, and submitting data	Our final specific topic is related to staff and grantee-level data. Our review indicated that programs sometimes needed guidance around this type of data, especially since it may not be used in your day-to-day program administration. For example, some programs needed assistance in understanding how to categorize staff who held multiple positions during the year, or understanding what activities are considered coaching. What, if any, challenges does your program experience related to collecting, maintaining and submitting: Workforce or staffing data (e.g., qualifications, turnover, salaries)? Coaching data? Collaboration agreements data? Prompt: How did you address these challenges? Prompt: Were there any challenges you were not able to address?
Understanding challenges related to collecting, maintaining, and submitting data	We want to make sure to provide space to hear about other challenges you are experiencing with the PIR data. What other data, if any, does your program find challenging to collect, maintain, and submit that we did not already discuss?
Understanding familiarity with OHS resources and supports	Now we’d like to return to Mentimeter for a few more questions. [Mentimeter slider-style question:] OHS offers several resources that programs can access about the PIR. How familiar are you with the following resources? [Slider response options from not at all familiar to extremely familiar] The Head Start Enterprise System help desk, which can be reached by phone or email for all PIR-related questions The PIR FAQ page on the ECLKC website, which lists answers to commonly asked PIR-related questions The PIR form PDF, which includes definitions and directions for completing the PIR The PIR change summaries, which list the changes made to the PIR from year-to-year The PIR XML documentation, which explains how the PIR reports should be formatted for uploading into HSES The PIR Reports User Guide, which explains how the PIR reporting system can be navigated to extract the data and view summary reports The PIR Performance Indicator Formulas, which explains how to calculate certain performance measures using PIR data Prompt: How do you prefer to receive support and resources from OHS? This could be through existing documentation or FAQs, or through new methods such as tips-and-tricks documents, webinars, or office hour sessions.
Understanding what support staff access related to PIR data	[Mentimeter slider-style question:] When you need help with a PIR-related question or challenge, how likely are you to access the following resources? [Slider response options from not at all likely to extremely likely] Head Start Enterprise System help desk PIR documentation (PIR form posted on HSES, PIR FAQ on EKCLC) My grantee’s regional office Other colleagues at my program or other programs MyPeers community members A person from my data management software company (such as ChildPlus, PROMIS, COPA) or other staff Online resources offered by my data management software (such as ChildPlus, PROMIS, COPA) None, I have not needed to access resources None, I am not sure how to access resources when I need them [Other] Prompt: How do you decide which resources to access when you have a challenge? This could be related to ease or convenience, or a feeling that you know where to get the most accurate and helpful information.	Suggest skipping if needed
Understanding what data is collected outside a program’s primary data system	This next question is about the data management software that your program uses to enter and manage service data, such as ChildPlus, PROMIS, or COPA. Some programs use more than one data management software, so we are asking specifically about the one your program uses most frequently. What data, if any, must be collected outside your main data management software (through Excel, another vendor’s system, etc.) because it is not collected by your data management software?	Suggest skipping if needed; this theme may naturally arise in the discussion on data challenges
Understanding use of PIR corrections	How often do you access the HSES after you have submitted your PIR data to update or correct the PIR data? Prompt: What topics required updates or corrections, and why?

Section 3: Looking Forward and Wrap-Up (15 minutes)

We have 15 minutes remaining together. As we wrap up, I’d like to ask some final questions about how to make the PIR data as useful as possible and leave some time for your final thoughts.

Topic	Central Office Questions and Prompts	Questions to skip if not enough time
Making PIR data useful	In a perfect world, what practices could the OHS implement to help your program collect, maintain, and submit high-quality PIR data? Prompt: Examples may include changing the submission schedule of the PIR, changing the reporting structure of PIR to focus on centers or individuals, or creating new practices that help programs collect and submit high-quality data. Prompt: Previously, we discussed some of the resources the OHS provides to programs to help them with the PIR, such as the HSES help desk, and the PIR form with definitions and directions. Based on our conversation today, what resources could OHS provide to you that would help you address questions or challenges that arise around the PIR submission? [Co-facilitator to enter practices into a Mentimeter “rankings” question as identified.] Which of these practices would be most helpful in improving the quality and usefulness of PIR data?	Suggest skipping this follow up if needed
Additional thoughts	Before we wrap up today, we want to leave a few minutes for anyone to share any thoughts about the PIR data that you haven’t had a chance to say yet. What final reflections do you have about the PIR as it relates to your program?

Those are all questions we have for today. We thank you all for taking the time to participate. We appreciate your insights and comments.

1