Instrument 22: Interview/Focus group protocol (administrator, teacher/assistant teacher and coach burden)

VIQI Interview/Focus Group Protocol

November 2020

Interview/Focus Group Protocol

The purpose of the Interview/Focus Group Protocol is to collect information on perceptions of the intervention and feedback on the surveys and logs. The interviews may be conducted with administrators, coaches, teachers and assistant teachers individually or with small groups, depending upon the number of individuals in each of the roles within a center and across centers in the same research condition. The data will be used to provide an early assessment of implementation of the interventions to inform how professional development, technical assistance and other forms of supports to centers, administrators, coaches and teachers should be adjusted in future phases of the study and to document the actual experiences of these staff in installing and delivering the interventions in classrooms and with children. The information will also be used to provide context to interpretation of findings from the pilot study and the impact evaluation and process study. Further the interviews will gather information about our data collection approaches and assess the degree of burden on participants, the extent to which the logs are capturing information as intended, and to inform changes to the impact evaluation and process studies.

[Note to reviewers: We plan to use different versions of the interview protocol to reduce the degree of burden on any individual participant. As such, an individual or group may only be asked to respond to questions from one or two of the major sections of the protocol: A) Experience with the VIQI Intervention Interview; B) Self-Survey Cognitive Interview; and C) Log Cognitive Interview. The specific questions/probes provided in this protocol are illustrative. Depending upon the flow and nature of the interview, different questions or follow-up questions may be asked to flesh out particular topical areas or ideas raised during the interviews, depending upon the responses of participants in the interviews.]

[INTERVIEWERS: PLEASE READ THE FOLLOWING TO PARTICIPANTS. MODIFY AS NECESSARY BASED ON THE CONTENT PLANNED FOR THE INTERVIEW OR FOCUS GROUP.]

Thank you for speaking with me today. I am a member of the research team for the VIQI study. VIQI is a national study of children’s experiences in early care and education across the country, with the goal of learning how centers can better support young children’s learning and growth. This project is funded by the U.S. Department of Health and Human Services

As part of the VIQI study, the research team is conducting on-site visits to centers. A critical component of these on-site visits are the focus groups and interviews, which will be used to gather feedback from staff on the intervention and data collection tools so they can be further refined in future phases of the study and to provide context for interpretation of study findings.

The purpose of this interview is to gather [INFORMATION ABOUT YOUR EXPERIENCE WITH THE INTERVENTION, COACHING AND TRAINING/FEEDBACK ON THE LOG, A REPORT ON YOUR CLASSROOM ACTIVITIES/COACHING ACTIVITIES THAT WE REQUEST FROM YOU EVERY [WEEK/UPON VISTING TEACHERS ON YOUR CASELOAD/FEEDBACK ON THE SELF-REPORTED SURVEY]. This interview/focus group is not an assessment or monitoring effort. Your participation is voluntary and you may skip questions you do not want to answer. Information you provide will be treated in a private manner and we will not attribute specific comments to individual respondents. Your feedback will help us refine the [intervention/teacher log /teacher survey] so that we are able to [CREATE AS STRONG AN INTERVENTION/COLLECT THE MOST USEFUL DATA] possible. This interview data will only be used in combination with the feedback from other study participants; we will not identify a specific person, center, or location. If you do not feel comfortable answering a particular question, please let me know and we will move on to the next one.

This interview/focus group will take about [90 MINUTES]. Do you have any questions?

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0508 and it expires XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to [Contractor Contact Name]; [Contractor Contact Address].

Section A: Experience with VIQI Intervention

Opening

1. What is your position? [Teacher, assistant teacher, master teacher, aide, etc.]

1. How long have you worked in this position?

2. How long have you worked at this center?

3. How long have you worked in the early childhood education and care field?

Perception of Intervention

1. What do you think of [name of curriculum intervention]?

1. What do you like about it? Not like? Please explain.

2. What aspects of [name of curriculum intervention] are most important?

3. What aspects of [name of curriculum intervention] are you focused on?

4. What changes have you made to your classroom/center, if any, as a result of VIQI?

2. What was the main factor influencing the change(s)?

Staff Roles

I/we want to get a sense of how you work with the other staff in your classroom to deliver [name of curriculum intervention].

1. Describe the role you have as [administrator/coach/lead or assistant teacher] in delivering [name of curriculum intervention].

   1. How does this compare to your role before VIQI?

2. Describe the role your [administrator/coach/lead/assistant teacher] has in delivering [name of curriculum intervention].

   1. How does this compare to his/her role before VIQI?

3. If roles have shifted, how has the change in roles affected your classroom/center? Relationship with the [lead/assistant teacher]?

   1. Do you feel the amount of instruction or quality of instruction has changed?

   2. Has the change has benefited your relationship with other staff? How so?

   3. Has the change been damaging to your relationship?

Coaching

1. Thinking about the coaching you have [delivered/received] during VIQI, what are the goals and topics of the coaching sessions?

   1. Examples: Setting up physical spaces in the classroom, arranging the daily schedule, choosing or preparing materials, understanding content matter being taught in activities, differentiating instruction, etc.

2. What feedback have you [received from your coach/given to teachers] about your teaching?

   1. By feedback, I/we mean comments or suggestions about your teaching, such as strengths and weakness, or modifications you can make to improve your teaching.

   2. Can you provide an example?

3. What do you/teachers need the most help with from the coach?

4. What do you spend the most time doing during coaching sessions trying to troubleshoot or problem-solve? E.g., addressing particular topics or activities from the intervention.

5. Please briefly describe what happened during the most recent coaching session?

   1. Where and when did it take place?

   2. Who was present?

   3. Did [your coach/you]: observe (doing what?); provide opportunities for practice; model; explain a concept or teaching strategy?

   4. What were the major topics focused on?

   5. To what extent does this last session represent a typical session?

   6. Do your coaching sessions vary a lot or do they tend to be the same in terms of timing and who is there? If they vary, what factors into the timing and attendance?

6. What about coaching has been most effective in supporting you/teachers with delivering [name of intervention]?

7. Were there parts of the coaching process that did not work well or could have been changed? Is there anything you would add?

Training

1. What do you/teachers need the most help with from training?

2. What about training has been most effective in supporting you/teachers with delivering [name of intervention]?

3. What could you/teachers use more training on?

4. Were there parts of training that did not work well or could have been changed? Is there anything you would add?

Barriers/Opportunities

1. Would you say [name of curriculum] is a good fit for you/your classroom/the classrooms in your center? If yes, how so? If not, what are the issues?

   1. Does it match your general philosophy or approach?

   2. Do the activities fit into your daily schedule?

   3. Do you have access to the resources you need? (Including materials and time to prepare).

   4. Is it relevant to the cultures and backgrounds of children and families in your classroom?

   5. Does it align with the other requirements you have from Head Start/funder?

2. What are the most challenging or difficult aspects of VIQI? Were there any unexpected challenges? What strategies have you used to address those unexpected challenges?

   1. Probe for challenges: Coordinating schedules for training and coaching, small groups, planning, setting up classroom, instructional quality practices, and different content areas: math, science, etc.

3. How prepared do you feel to [deliver/support teachers in delivering] the [name of curriculum intervention]? What could have been helpful/supportive to you in the delivery of [name of curriculum]?

4. FOR TEACHERS/ASSISTANT TEACHERS: How is your [administrator/director/educational director] supporting you to deliver [name of curriculum intervention]?

   1. Is it important to your [administrator/director/educational director] that you are teaching [name of curriculum intervention]?

5. FOR ADMINISTRATORS/COACHES: How are you supporting teachers to deliver [name of curriculum intervention]?

   1. Is it important to you that your teachers deliver [name of curriculum intervention]?

6. Do you think there were opportunities to enhance your understanding and delivery of [name of curriculum] that we might have missed? For example, ongoing activities, trainings, or professional development, or staff that typically occur in your center that we might have overlooked?

Reflection

1. Now that you’ve had experience with [name of curriculum intervention], coaching, and training for VIQI, is there anything you wish you would have known from the beginning?

   1. Did you get enough information about VIQI before the fall?

   2. Were all of your questions about VIQI answered?

   3. How could your experience be improved?

2. In your opinion, what has been most valuable aspect of being part of VIQI? What do you like most about VIQI?

Section B: Survey Cognitive Interview

Introduction: Now we’re going to discuss your experiences completing the survey. Please reference the surveys that we’ve passed around. These are the same surveys [administrators/coaches/teachers/assistant teachers] were asked to complete in spring [year]. I/we will be asking for your feedback on the questions that were included in the survey. The feedback will be helpful as we continue to refine the survey instrument and methodology so that we are able to collect the most useful data possible.

1. Please tell me about your overall experience completing the survey.

1. How did you complete the survey – web or paper? What is your preference, and why?

2. Overall, were some sections easier or more difficult for you to complete? If so, which ones were easier or more difficult? Why?

2. Please tell me about your overall experience completing the [SECTION NAME].

1. What was your general reaction to this section?

2. Was the phrasing of items in this section clear and easy to understand? Please let me know if any questions were not clear to you, or did not seem relevant to you.

3. Did the terms used [SPECIFY TERMS, AS NECESSARY] resonate with you?

4. Did you have any trouble deciding which response option you should select for an item? Were any of the response options difficult to distinguish?

5. Was there a [CATEGORY/TOPIC/ACTIVITY] that didn’t get included?

6. Do any of the items feel like they are asking about the same thing?

7. Is there additional information or directions we could provide to improve the experience?

[INTERVIEWERS: WALK THROUGH EACH SECTION OF THE SURVEY TO GATHER FEEDBACK ON THE ITEMS.]

Section C: Log Cognitive Interview

Introduction: Now we’re going to discuss your experiences completing the log. Please reference the paper copies of the log that we’ve passed around. These are the same logs [coaches/teachers/assistant teachers] are asked to complete on a [after each session with teachers/weekly] basis. I/we will be asking for your feedback on the questions that were included in the teacher log. The feedback will be helpful as we continue to refine the teacher log instrument and methodology so that we are able to collect the most useful data possible.

1. Please tell me about your overall experience completing the log.

1. About how long does it take you to complete each log?

2. Overall, were some sections easier or more difficult for you to complete? If so, which ones were easier or more difficult? Why?

2. Please tell me about your process for completing the log.

   1. What day/time works best?

   2. Do you complete it all at once or take multiple days to complete it?

   3. What do you do when you aren’t involved in a particular activity, like if another teacher delivered the large or whole group activities and you weren’t involved?

3. Please tell me about your overall experience completing the [SECTION NAME].

1. What was your general reaction to this section?

2. Was the phrasing of items in this section clear and easy to understand? Please let me know if any questions were not clear to you, or did not seem relevant to you.

3. Did the terms used [SPECIFY TERMS, AS NECESSARY] resonate with you?

4. Did you have any trouble deciding which response option you should select for an item? Were any of the response options difficult to distinguish?

5. Was there a [CATEGORY/TOPIC/ACTIVITY] that didn’t get included?

6. Do any of the items seem like they are asking about the same thing?

7. Is there additional information or directions we could provide to improve the experience?

[INTERVIEWERS: WALK THROUGH EACH SECTION OF THE SURVEY TO GATHER FEEDBACK ON THE ITEMS.]

[INTERVIEWERS: ASK THE FOLLOWING QUESTIONS ABOUT THE “TEACHER PRACTICE” ITEMS (TEACHER LOG: 6; 12; 24; 34; 43; COACH LOG: 8; 16; 25; 32; 38)

4. Please take a look at the list of teaching practices on page [#].

   1. Can you provide me with an example of [SPECIFY TEACHING PRACTICE FROM ITEM]?

      1. Probe for details as needed and repeat process with multiple teaching practices.

   2. Do any of the teaching practices on the list seem to be asking about the same thing?

   3. Was there a teaching you often use/observe that didn’t get included?

8