Attachment B: Semi-Structured Protocol for VIQI Landscaping Discussions
State/Local Version
Guidelines for Use of this Protocol in Landscaping Discussions for the Variations in Implementation of Quality Interventions (VIQI) Study. This protocol will be used to guide a series of semi-structured landscaping discussions with state and local child care and early education (CCEE) informants. In each module, illustrative questions and prompts provide examples of the types of questions that will be asked.
Modules
will cover the following topics:
Module
A. CCEE Program/Center Structure and Population Served
Module B.
Quality and Quality Measures
Module C. Curricula and Professional Development
Module D. Data Infrastructure
Module
E. Feasibility of Design Options
Module F. Follow-up/Next
Steps
Facilitators
will not cover all of the questions in this protocol during every
discussion.
We are asking informants for a discussion that is expected to last
approximately one hour. We may also conduct a second discussion to
follow-up and clarify information emerging
from the initial discussion. In total, we expect that any given
participant will be engaged in a maximum of 1.5 hours of discussion.
Facilitators
will select and edit questions/prompts from different modules as
needed, depending on: 1) the expertise of the informants with whom
they are speaking; and 2) the study team’s extant knowledge
base accumulated through earlier discussions. Additional
questions and prompts (not included in this protocol) may be asked as
non-standardized follow-up prompts and to clarify responses provided
by informants.
Members of the VIQI study team will serve as facilitators, and will:
Introduce members of the study team and ask informants to introduce themselves and describe their roles and responsibilities.
Provide a brief overview of the VIQI study and summarize the purpose of the call/visit. (See Attachment E for agenda).
Standard introductory language to be used by facilitators (not for distribution):
The Administration for Children and Families’ Office of Planning, Research and Development is working with MDRC and its partners on a major new study that will examine how different dimensions and levels of quality in early childhood classrooms relate to children’s developmental outcomes. We plan to partner with multiple localities to conduct this study.
In the initial planning stages for this study, we are reaching out to leaders in the early childhood field to:
share more details about the study;
ask questions to help us better understand the landscape of CCEE services; and
discuss potential opportunities and challenges that our study team should consider in planning for this study.
Participation is voluntary, and responses will be kept private to the extent permitted by law.
An
agency may not conduct or sponsor, and a person is not required to
respond to, a collection of information unless it displays a
currently valid OMB control number. The OMB number for this
information collection is 0970-0356 and the expiration date is
03/31/2018.
Answer clarifying questions about the purpose of the discussion and study plans.
Select
questions and prompts from the following modules as needed based on
the expertise of the informants
Module A. CCEE Program/Center Structure and Population Served. This module focuses on the structural characteristics of CCEE services in specific localities. Illustrative questions and possible prompts are provided below. Note: Some questions and prompts in this module also appear in Module D (“Data Infrastructure”), but the prompts serve different purposes across the two modules. In Module A, the prompts serve to gather high-level, first-hand insights from experts regarding the landscape of CCEE services in different localities. In Module D, the prompts serve to learn more about the type and depth of information being stored in existing data sources. Asking the prompts in Module D allows us to identify specific data sources that might be helpful to explore to develop a more detailed understanding about the characteristics and publicly-funded child care programs and centers nationally and in different localities.
Introduction
to Module A. The
VIQI study will be conducted in collaboration with multiple
localities. By “locality,” we mean “metropolitan
areas.” We will be looking for localities
where there are large numbers of Head Start and publicly-funded child
care centers serving children from low-income backgrounds, and where
there is a documented wide range of quality among centers across
settings. In this initial planning phase, we would like to develop a
better understanding of the landscape
of CCEE services in [locality X], including the structural
characteristics of CCEE programs and centers, and the populations of
children and families served.
Illustrative Questions and Prompts
Who are the key stakeholders involved in oversight of publicly-funded child care and Head Start programs in [locality X]?
Possible prompts:
What are the major nonprofit organizations that operate multiple CCEE centers?
Who determines which curricula are used?
Who determines what professional development teachers receive?
Who
are the other key decision-makers or influencers? (e.g., funders,
teachers’ unions, heads of early childhood coalitions or
advocacy groups)
Are there large concentrations of certain types of CCEE programs in [locality X]?
Possible
prompts:
large,
multi-site non-profit organizations, for-profit organizations,
faith-based organizations
What is the age make-up of most CCEE classrooms?
Possible prompts:
Are two’s, three’s, and four’s in mixed age classrooms or separated based on birth date?
Is this different across different CCEE settings? (Head Start, publicly-funded child care?)
Are
three’s included in public pre-k programs?
We need to develop a detailed understanding of the characteristics of Head Start and publicly-funded child care programs and centers in [locality X]. Do you have answers to the following questions, or can you tell us how to obtain the answers?
Number of Head Start and publicly-funded child care programs and centers
Number of centers per program
Number of classrooms per center
Length of day (What percentage are half-day vs. full-day?)
Demographic data about children served by CCEE programs and centers in a given locality
The number of children ages two, three, and four
The proportion of children who are from low-income families
The proportion of children who are English language learners
The proportion of children who have special needs
Module B. Quality and Quality Measures. This module aims to understand how states and localities might assess and measure CCEE quality on an ongoing basis. It further aims to understand how this information is stored and maintained. Note that some of the questions in this module will overlap with the questions in Module D (Data Infrastructure), but this module is designed to focus the discussion on measures of quality, while Module D is designed to gather information about a broader set of data that can inform the VIQI study. Illustrative questions and possible prompts are provided below.
Introduction
to Module B.
We
would like to ask for your insights about common ways in which the
quality of CCEE programs and centers is described and measured, and
about state/local trends in quality improvement.
Illustrative Questions and Prompts
The field has identified several basic dimensions of classroom quality that are hypothesized to influence children’s outcomes – namely structural quality (how centers and classrooms are designed and configured) , process quality (quality of children’s interactions with teachers and others in the classroom, including warmth, sensitivity, and overall classroom management and organization, and instructional quality (in terms of intentional teaching of children through organized activities, including scope and sequence of activities within specific domains).
Possible
prompt: Are there other aspects of quality that locality X (or the
state) is particularly interested in?
What are important recent, current, or upcoming initiatives to improve CCEE quality at the state and local levels?
Possible prompts:
Any major recent/upcoming changes to CCEE curricula?
Other
initiatives led by foundations or regional/state/local early
childhood coalitions?
Which extant data sources (Quality Rating and Improvement Systems [QRIS], other) can be used to determine levels of quality among CCEE programs and centers in [locality X]?
Possible prompts:
What data/management information systems (MIS) are used to facilitate monitoring of child care and Head Start programs and centers in [locality X]?
Tracking as part of community-based organizations’ services
Tracking as a part of a local CCEE initiative
Other tools and measures used to monitor curriculum implementation
How would you describe the variation in levels of quality among CCEE programs (and centers within those programs) that are included in these monitoring systems? Are there many high AND low quality classrooms?
What about the programs and centers that are NOT included in these monitoring systems?
How would you describe the variation in levels of quality by type of CCEE program (e.g. Head Start, publicly-funded child care center)?
What are the reasons for the variation in quality?
Module C. Curricula and Professional Development. This module focuses on common curricula, curricular enhancements, professional development (PD) initiatives, and other interventions that [locality X] is using in Head Start and publicly-funded child care centers. Illustrative questions and possible prompts are provided below.
Introduction to Module C. In the planning and design phase of the VIQI study, we are gathering information about the curricula, curricular enhancements, professional development initiatives, and other interventions that localities are using with three and four year-olds in Head Start and publicly-funded child care centers. We would like to ask for your thoughts about this topic, and any advice you can offer regarding other stakeholders or resources that might help us with this effort.
Illustrative
Questions and Prompts
Curricula and curricular enhancements
What curricula and curricular enhancements are commonly used within Head Start centers in [locality X]? within publicly-funded child care centers?
Possible prompts:
Different curricular/interventions for ages two, three and four?
Curricular
enhancements focused on particular domains (e.g., math, language
and literacy, social-emotional skills)?
What factors were considered when choosing these curricula/interventions?
Possible prompts:
Ages of children?
Evidence base?
Professional development model?
Alignment with other existing standards?
Cultural
context?
[For
State-level stakeholders] How does choice of intervention/curricula
vary from locality to locality and why?
Professional Development (PD)
We
are exploring how professional development (or PD) might influence
the level of quality in a CCEE center or classroom. What are
important recent, current, or upcoming initiatives that focus on
teacher PD as a way to improve quality in [locality X]?
What does in-service training for CCEE teachers entail?
In Head Start? In publicly-funded child care settings?
How
often, how long, and on what topics?
Do teachers receive mentoring or coaching?
How often, how long, by whom, and on what topics?
Does
it involve a classroom visit and/or a sit-down meeting?
Do teachers have shared planning time?
How
often, how long, and with whom?
What are some of the challenges that states and localities face with respect to PD for CCEE teachers?
Module D. Data Infrastructure. This module is designed to explore the types of data collected on the state and local levels that can inform the VIQI study. Illustrative questions and possible prompts are provided below. Note: Some questions and prompts in this module also appear in Module A (“CCEE Program/Center Structure and Population Served”), but the prompts serve different purposes across the two modules. In Module A, the prompts serve to gather high-level, first-hand insights from experts regarding the landscape of CCEE services in different localities. In Module D, the prompts serve to learn more about the type and depth of information being stored in existing data sources. Asking the prompts in Module D allows us to identify specific data sources that might be helpful to explore to develop a more detailed understanding about the characteristics and publicly-funded child care programs and centers nationally and in different localities.
Introduction to Module D. One of the goals of the VIQI study is to understand the effects of different dimensions and thresholds of quality on child outcomes and to understand how these effects might vary depending upon the characteristics of the participating CCEE programs and the populations they serve. In the pilot and full-scale phases of this study, we would like to work with localities that have a large number of CCEE centers that represent a wide range of quality. We would like to understand, from your perspective, what types of data already exist that might help our team to better understand the landscape of these programs (and centers within those programs) in [locality X].
Illustrative Questions and Prompts
What data sources do you think would be helpful in this endeavor? (At the state level? At the local level?)
Possible prompts:
Quality Rating and Improvement Systems (QRIS)
Monitoring data collected by local agencies with oversight of CCEE programs
Are
there any initiatives or efforts underway to build a CCEE data
infrastructure?
Possible prompts for each data source mentioned:
Does this data source include names and characteristics of Head Start or publicly-funded child care centers?
At what level: Individual centers? Programs?
Does the data source provide information on the following features of programs, centers and population characteristics?
Structural characteristics
Number of centers in a given locality
Number of classrooms per center
Length of day (what percentage of centers are half-day vs. full-day?)
Grouping of two-four year olds: are they usually in mixed classrooms, or segregated by age?
Demographic data about children served by CCEE programs in [locality X]
The number of children ages two, three, and four
The proportion of children who are from low-income families
The proportion of children who are English language learners
The proportion of children who have special needs
Characteristics of administrators, teachers, and other staff
By center or program
Curricula and Professional Development
Curricula and curricular enhancements used by individual centers
Information about professional development (how long, how often, on what topic, etc.)
How are these data maintained?
How frequently are the data updated in this data source?
What is the catchment area covered by this data source?
What is the level of penetration for this data source?
Does the level of penetration differ for Head Start or publicly-funded child care programs?
Is participation in this data source voluntary or mandatory?
How complete are these data? Are there concerns about missing fields of information in this data source? What might these missing fields be?
How accurate is the information included in the data source?
How are these data maintained?
How can these data sources be accessed? What is the process for accessing the data?
Are CCEE management information systems (MIS) within localities usually linked to each other? To other MIS systems (e.g., k-12)? If so, for what purposes?
Possible prompt: Are there unique IDs that are used to follow children’s progress from child care through the K-12 system?
How are MIS data structured to describe quality (e.g., at the center or program level? At the state or local level?)
What are the primary measures that are used to assess classroom quality, teacher quality, and child outcomes in CCEE programs and centers in [locality X]?
Possible prompts:
What are the key dimensions of these measures?
On classroom quality, is there anything collected besides the ECERS or the CLASS?
Module E. Feasibility of Design Options. The questions in this module are designed to elicit additional advice from informants about any topics or issues that have not already been covered in the discussion. Illustrative questions and possible prompts are provided below.
Introduction
to Module E. We
would like to know whether there are any other words of advice you
might have for us as we embark on this study – including any
additional thoughts about opportunities or challenges we should have
on our radar screen, or any other questions you think we should
consider.
Illustrative Questions and Prompts
We
are considering a focus on three year-olds in center-based care. We
would appreciate your thoughts about focusing on this age group.
What are the opportunities/challenges that would come with a focus
on threes?
We have a particular interest in recruiting a sample of CCEE programs and centers that includes good representation of programs and centers that are both high and low in initial quality to stratify the sample. What opportunities/challenges in [locality X] would come with trying to do so?
Possible prompts:
How might the existing distribution of initial quality across CCEE programs and centers affect our ability to do so?
Might this vary by setting type?
Might this vary specifically for CCEE programs and centers serving 3 year olds?
What are your thoughts about the challenges we may encounter (either in site recruitment, or implementation of quality interventions)?
Possible prompt:
Do
you have any advice for us regarding possible solutions to those
challenges?
Is [locality X] (or the CCEE programs/centers within [locality X]) participating (or planning to participate) in any other research projects?
Module F. Follow-up/Next Steps. This module focuses on wrapping up the discussion and laying out next steps. Illustrative questions and possible prompts are provided below.
Introduction to Module F. In the last few minutes of this discussion, we would like to discuss next steps, make plans to follow up on items that we have discussed today, and address any other questions you might have for us.
Illustrative
Questions and Prompts
Are there other particular localities that you think might have the following characteristics:
A range of program quality
Large numbers of Head Start and publicly-funded child care centers serving children from low-income backgrounds
Localities
that have strong commitment to data collection and quality
Do
you have suggestions for key contacts in these localities?
We want to make sure we speak with the people who can best inform our landscaping and study design. Based on the nature of our questions and interests, who else would you suggest we speak with to inform the study?
Possible prompts:
Other stakeholders in the CCEE system
Other key staff in state/local child care/Head Start programs
Experts on curricula and interventions, quality and quality improvement, professional development
Experts
on CCEE data infrastructure
(If
appropriate):
If a follow-up call or visit is needed, discuss timelines and
identify a point person for follow-up.
(If
appropriate):
Would your office be willing to sign a letter of introduction
addressed to (insert names of stakeholders as appropriate)
describing the VIQI study and asking them to speak with someone from
our team about the VIQI study?
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
Author | Jan DeCoursey |
File Modified | 0000-00-00 |
File Created | 2021-01-13 |