Early Childhood Training and Technical Assistance (T/TA) Cross-System Evaluation Project

Early Childhood T/TA Cross-System Evaluation Project

Appendix D: Discussion Guide for Focus Groups on Draft Conceptual Framework

INTRODUCTION: Thank you for taking time to meet with us. As we explained in our email, we contacted you on behalf of the Administration for Children and Families (ACF), which in partnership with us at NORC is designing and implementing a utilization-focused evaluation for the specific purpose of documenting, assessing, and enhancing ACF’s Early Childhood Training and Technical Assistance (EC T/TA) System. The goal of this focus group is to present a conceptual framework of the EC T/TA System and then to obtain your feedback on the model, which addresses key stakeholders, T/TA components, intended beneficiaries, and outcomes. We are at a point where we want to obtain feedback to revise and refine a draft of the conceptual framework. We started by reviewing Funding Opportunity Announcements, ACF documents and webinar presentations, existing work plans of the National Centers, and other publically available materials about early childhood T/TA delivery. Development of the conceptual framework has also been informed by a review of the literature. During this focus group, we will ask questions intended to check the representation of stakeholders, components, relationships, processes, and outcomes in the proposed model, as well as identify areas for improvement of the model. The focus group should take approximately 90 minutes. Your open and honest opinions are appreciated.

Participation is voluntary. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0356 and the expiration date is 03/31/2018.

Please know that all the information you provide in these discussions will be kept private to the extent permitted by law. In order to capture your responses accurately, we will be recording today’s discussion for our reference and will destroy the file once a transcript is made.

Presentation of the Conceptual Framework.

To begin, a member of the NORC research team will present a PowerPoint slide which depicts the conceptual framework of the Early Childhood T/TA System. The presentation will address how it was developed and its purpose for the utilization-focused evaluation. A written statement will also be provided to ensure that the presentation is 508 compliant. Following the presentation and explanation of the conceptual framework, including how it was developed and its purpose, a researcher will facilitate a discussion to gather feedback and perceptions from the participants by drawing on the following questions and asking for clarification on participants’ comments, as appropriate.

Introduction

Now that you have heard about the purpose and the contents of this conceptual framework, we are eager to hear from you about your reactions to this portrait of the training and technical assistance system which you are an integral part. Your comments, suggestions and ideas will help ensure that this description of the system is as accurate as possible in order to guide the evaluation design.

1. Components of the EC T/TA System

We want to begin the discussion today by reviewing this portrait of the EC T/TA System components and stakeholders.

1. Does this diagram accurately depict the multiple components of ACF’s Early Childhood T/TA system?

   1. Is your role and its relationship with other components of the T/TA system reflected accurately in the diagram?

   2. Does the model include all the major T/TA providers in the system? [Probes: Significant partners? T/TA recipients or beneficiaries? Other key stakeholders in the system?]

   3. Should any of these stakeholders be more explicit? [Probes: Such as the Primary T/TA recipients? States and Territories that are engaged in Impact Projects and their partners?

2. Does the diagram accurately capture the T/TA resources and methods used within the system?

   1. Are there any important components missing that should be included to depict what occurs within the T/TA system?

   2. Does the depiction of T/TA delivery as an iterative cycle involving tools, training, technical assistance, and quality improvement/quality assurance capture the way in which T/TA is delivered?

2. Relationships and Processes

Next, we want to focus on the relationships and processes described in the model and see if they are appropriately depicted.

3. Does this model accurately depict the relationships within ACF’s Early Childhood T/TA System

4. Does the diagram depict the processes and inner-workings of the T/TA system as they occur?

5. Does the diagram present the primary pathways of information dissemination from the T/TA providers to the intended recipients or beneficiaries?

6. Does this conceptual framework provide sufficient detail to inform multiple stakeholders (such as ACF, national centers, regional networks, states and territory lead agencies, tribal grantees, Head Start and Early Head Start grantees, EHS-Child Care Partnership grantees, and partner organizations) about the components and relationships that make up the Early Childhood T/TA system?

   1. Does the diagram allow you to see the big picture view?

   2. Is the diagram too detailed? If so, how or why?

   3. Should we simplify the diagram? If so, how or why?

   4. Would it help to go into greater depth or detail in any areas?

7. Are there any organizational factors that should be added or changed?

8. Are there any contextual factors that should be added or changed?

9. Should the interaction of implementation drivers with T/TA delivery be made more explicit in the diagram (i.e., leadership, staff competency and coaching, system supports)?

10. Are there other factors shaping the provision and receipt of T/TA that we should represent in the model?

3. Outcomes

Next, we want to check the outcomes described in the model to confirm that they represent the intended effects for each entity and are properly aligned.

11. Does the model capture the intended outcomes of the T/TA system as you expect or see them occur?

    1. Are the outcomes in the diagram:

       • Presented at the right level?

       • Accurate?

       • Clear?

       • Specific enough?

       • Realistic?

       • Measurable?

4. Areas for Additional Description

Finally, we want to hear your ideas about which topics or areas are not described enough in the model.

12. Having reviewed each component of the conceptual framework, are there any constructs or topics that should be included based on your experience or knowledge of the field?

13. Do you have any other thoughts or ideas for how to improve this depiction of ACF’s Early Childhood T/TA System?

CLOSING: Thank you for participating in this discussion today. We appreciate your willingness to generously share your expertise and time with us. If you have any additional information, comment or questions, please do not hesitate to email or call us at [INSERT FACILITATOR CONTACT INFORMATION HERE].