Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education

Attachment A: Semi-Structured Protocol for VIQI Landscaping Discussions

National/Regional Version

Guidelines for Use of this Protocol in Landscaping Discussions for the Variations in Implementation of Quality Interventions (VIQI) Study. This protocol will be used to guide a series of semi-structured landscaping discussions with national and regional child care and early education (CCEE) informants. In each module, questions and prompts are provided as examples of the types of questions that will be asked.

Modules will cover the following topics:

Module A. CCEE Program/Center Structure and Population Served
Module B. Quality and Quality Measures

Module C. Curricula and Professional Development

Module D. Data Infrastructure

Module E. Feasibility of Design Options
Module F. Follow-up/Next Steps

Facilitators will not cover all of the questions in this protocol during every discussion. We are asking informants for an initial discussion that is expected to last approximately one hour. We may also conduct a second discussion to follow-up and clarify information emerging from the initial discussion. In total, we expect that any given participant will be engaged in a maximum of 1.5 hours of discussion. Facilitators will select and edit questions/prompts from different modules as needed, depending on: 1) the expertise of the informants with whom they are speaking; and 2) the study team’s extant knowledge base accumulated through earlier discussions. Additional questions and prompts (not included in this protocol) may be asked as non-standardized follow-up prompts and to clarify responses provided by informants.

Members of the VIQI study team will serve as facilitators, and will:

1. Introduce members of the study team and ask informants to introduce themselves and describe their roles and responsibilities.

2. Provide a brief overview of the VIQI study and summarize the purpose of the call. (See Attachment E for agenda.)
   
   Standard introductory language to be used by facilitators (not for distribution):

• The Administration for Children and Families’ Office of Planning, Research and Development is working with MDRC and its partners on a major new study that will examine how different dimensions and levels of quality in early childhood classrooms relate to children’s developmental outcomes. We plan to partner with multiple localities to conduct this study.

• In the initial planning stages for this study, we are reaching out to leaders in the early childhood field to:

• share more details about the study;

• ask questions to help us better understand the landscape of CCEE services; and

• discuss potential opportunities and challenges that our study team should consider in planning for this study.

• Participation is voluntary, and responses will be kept private to the extent permitted by law.

• An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0356 and the expiration date is 03/31/2018.

3. Answer clarifying questions about the purpose of the discussion and study plans.

4. Select questions and prompts from the following modules as needed based on the expertise of the informants
   
   
   

Module A. CCEE Program/Center Structure and Population Served. This module focuses on the structural characteristics of CCEE services. Illustrative questions and possible prompts are provided below. Note: Some questions and prompts in this module also appear in Module D (“Data Infrastructure”), but the prompts serve different purposes across the two modules. In Module A, the prompts serve to gather high-level, first-hand insights from experts regarding the landscape of CCEE services in different localities. In Module D, the prompts serve to learn more about the type and depth of information being stored in existing data sources. Asking the prompts in Module D allows us to identify specific data sources that might be helpful to explore to develop a more detailed understanding about the characteristics and publicly-funded child care programs and centers nationally and in different localities.

Introduction to Module A. The VIQI study will be conducted in collaboration with multiple localities. By “locality,” we mean “metropolitan areas,” as we will be looking for localities where there are large numbers of Head Start and publicly-funded child care centers serving children from low-income backgrounds, and where there is a documented range of quality among centers across settings. In this initial planning phase, we are seeking insights from stakeholders in the field that will help us develop a better understanding of the landscape of CCEE services in different localities, including the structural characteristics of CCEE programs and centers, and the populations of children and families served.

Illustrative Questions and Prompts

• What is your advice about the most efficient ways for us to develop a detailed understanding of characteristics of Head Start and publicly-funded child care programs and centers in different localities, including:

• Number of Head Start and publicly-funded child care programs and centers in a given locality

  • Number of centers per program

  • Number of classrooms per center

• Length of day (what percentage are half-day vs. full-day)

  • Does the length of day tend to vary greatly from locality to locality?

  • If yes, what influences this variation?

• Demographic data about children served by CCEE programs and centers in a given locality

• The number of children ages two, three, and four

  • The proportion of children who are from low-income families

  • The proportion of children who are English language learners

  • The proportion of children who have special needs

• What is the age make-up of most CCEE classrooms?

  • Are two’s, three’s, and four’s in mixed age classrooms or separated based on birth date?

  • Is this different across different CCEE settings? (Head Start, publicly-funded child care?)

  • Are three’s included in public pre-k programs?
    
    

• Are there particular localities that are known to have large concentrations of certain types of CCEE programs?

  • Possible prompt: large multi-site non-profit organizations, for-profit organizations, faith-based organizations

Module B. Quality and Quality Measures. This module focuses on the ways in which CCEE quality is assessed on an ongoing basis. It further asks about current initiatives to improve CCEE quality. Illustrative questions and possible prompts are provided below.

Introduction to Module B. We would like to ask for your insights about common ways in which the quality of CCEE programs/centers is described and measured, and about national and regional trends in quality improvement.

Illustrative Questions and Prompts

• The field has identified several basic dimensions of classroom quality that are hypothesized to influence children’s outcomes – namely structural quality (how centers and classrooms are designed and configured) , process quality (quality of children’s interactions with teachers and others in the classroom, including warmth, sensitivity, and overall classroom management and organization, and instructional quality (in terms of intentional teaching of children through organized activities, including scope and sequence of activities within specific domains).

  • Are there other aspects of quality that the CCEE field is particularly interested in? Given your expertise, what other features of CCEE programs/centers/classrooms do you think are driving high vs. low quality?
    
    

• Other than the CLASS and/or ECERS, what are some common measures that are used by different localities to assess classroom quality?

  • Possible prompt: What dimensions of quality are included in those measures?
    
    

• What is the most effective way to understand the range of quality among Head Start and publicly-funded child care centers in various localities?

  • Possible prompts:

    • State data tracking systems

    • Local monitoring efforts by Head Start grantees and child care programs or centers
      
      

• What are important recent, current, or upcoming initiatives to improve quality at the national and regional levels?

  • Possible prompts:

    • Any major recent/upcoming changes to CCEE curricula?

    • Other initiatives led by foundations or national/regional CCEE coalitions?

Module C. Curricula and Professional Development. This module focuses on common curricula, curricular enhancements, professional development (PD) initiatives, and other interventions that localities are using in Head Start and publicly-funded child care centers. Illustrative questions and possible prompts are provided below.

Introduction to Module C. In the planning and design phase of the VIQI study, we are gathering information about the curricula, curricular enhancements, professional development initiatives, and other interventions that localities are using with three and four year-olds in Head Start and publicly-funded child care centers. We would like to ask for your thoughts about this topic, and any advice you can offer regarding other stakeholders or resources that might help us with this effort.

Illustrative Questions and Prompts

• We know that Creative Curriculum and High Scope are very commonly used. Are there any organizations or data sources that track curriculum use? (Internal note: OPRE may be able to do analyses of the NSECE data to assess patterns across programs, and PIR data from OHS contains information about this at the grantee/delegate agency level).
  
  

• How does the national or regional context influence the choice of and/or implementation of interventions and curricula?

  • Possible prompts:

    • Hours when children are in care

    • Cultural context

    • Language

    • Local educational expectations for kindergarten

    • Percentage of three year-olds vs. four year-olds
      
      

• We are exploring how professional development (or PD) might influence the level of quality in a CCEE center or classroom. What are important recent, current, or upcoming initiatives that focus on teacher PD as a way to improve quality?
  
  

• What are some of the challenges that localities face with respect to professional development for CCEE teachers?

Module D. Data Infrastructure. This module is designed to explore the types of data collected on the national, regional and local levels that can inform the VIQI study. Illustrative questions and possible prompts are provided below. Note: Some questions and prompts in this module also appear in Module A (“CCEE Program/Center Structure and Population Served”), but the prompts serve different purposes across the two modules. In Module A, the prompts serve to gather high-level, first-hand insights from experts regarding the landscape of CCEE services in different localities. In Module D, the prompts serve to learn more about the type and depth of information being stored in existing data sources. Asking the prompts in Module D allows us to identify specific data sources that might be helpful to explore to develop a more detailed understanding about the characteristics and publicly-funded child care programs and centers nationally and in different localities.

Introduction to Module D. One of the goals of the VIQI study is to understand the effects of different dimensions and thresholds of quality on child outcomes and to understand how these effects might vary depending upon the characteristics of the CCEE programs and centers and the populations they serve. In the pilot and full-scale phases of this study, we would like to work with localities that have a large number of CCEE centers that represent a wide range of quality. We would like to understand, from your perspective, what types of data already exist that might help our team to better understand the landscape of these programs (and the centers within these programs) at the local level.

Illustrative Questions and Prompts

• What data sources do you think would be helpful in this endeavor? At a national level? At a state level? At a locality level (e.g., city, metropolitan area)?

  • Possible prompts:

    • Quality Rating and Improvement Systems (QRIS)

    • Program Information Reports, Quality Progress Reports

    • Other data sources
      
      

• Are there particular states or localities where these data sources are actively maintained, and have good penetration or coverage?

  • If yes: What are these states or localities?

  • If no: Are there states or localities where initiatives or efforts are underway to build data infrastructure or sources?
    
    

• Possible prompts for each data source mentioned:

  • Does this data source include names and characteristics of Head Start or publicly-funded child care centers?

    • At what level: Individual centers? Or programs?

  • Does the data source provide information on the following features of CCEE programs, centers and population characteristics?

    • Structural characteristics

      • Number of centers in a given locality

      • Number of classrooms per center

      • Length of day (What percentage of centers are half-day vs. full-day?)

    • Information about how two-four year olds are grouped: are they usually in mixed classrooms, or segregated by age?

    • Demographic data about children served by CCEE programs and centers in a given locality

      • The number of children ages two, three, and four

      • The proportion of children who are from low-income families

      • The proportion of children who are English language learners

      • The proportion of children who have special needs

    • Characteristics of administrators, teachers, and other staff

      • By center or program

    • Curricula and Professional Development

      • Curricula and curricular enhancements used by individual centers

      • Information about professional development (how long, how often, on what topic, etc.)

  • How are these data maintained?

    • How frequently are the data updated in this data source?

    • What is the catchment area covered by this data source?

    • What is the level of penetration for this data source?

      • Does the level of penetration differ for Head Start or publicly-funded child care programs/centers?

      • Is participation in this data source voluntary or mandatory?

    • How complete and accurate are these data? Are there concerns about missing fields of information in this data source? What might these missing fields be?

    • How can these data sources be accessed? What is the process for accessing the data?

    • Are CCEE management information systems (MIS) within localities usually linked to each other? To other MIS systems (e.g., k-12)? If so, for what purposes?

Module E. Feasibility of Design Options. The questions in this module are designed to elicit a broad variety of additional advice from informants about any topics or issues that have not already been covered in the discussion. Illustrative questions and possible prompts are provided below.

Introduction to Module E. We would like to know whether there are any other words of advice you might have for us as we embark on this study – including any additional thoughts about opportunities or challenges we should have on our radar screen, or any other questions you think we should consider.

Illustrative Questions and Prompts

• We are considering a focus on three year-olds in center-based care. We would appreciate your thoughts about focusing on this age group. What are the opportunities/challenges that would come with a focus on threes?
  
  

• We have a particular interest in recruiting a sample of CCEE programs and centers that includes good representation of programs and centers that are both high and low in initial quality to stratify the sample. What opportunities/challenges in [locality X] would come with trying to do so?

  • Possible prompts:

    • How might the existing distribution of initial quality across CCEE programs and centers affect our ability to do so?

    • Might this vary by setting type?

    • Might this vary specifically for CCEE programs and centers serving 3 year olds?

• What are the key factors and characteristics of localities that might make it easier or harder to implement quality interventions?

  • Possible prompt: demographics of population served, CCEE governance, funding structure?
    
    

• What are your thoughts about the challenges we may encounter (either in site recruitment or implementation of quality interventions)?

  • Possible prompt: Do you have any advice for us regarding possible solutions to those challenges?

Module F. Follow-up/Next Steps. This module focuses on wrapping up the discussion and laying out next steps. Illustrative questions and possible prompts are provided below.

Introduction to Module F. In the last few minutes of this discussion, we would like to discuss next steps, make plans to follow up on items that we have discussed today, and address any other questions you might have for us.

Illustrative Questions and Prompts

• Other than the localities we have already discussed, are there other particular localities that you think might have the following characteristics:

  • A range of program quality

  • Large numbers of Head Start and publicly-funded child care centers serving children from low-income backgrounds

  • Localities that have strong commitment to data collection and quality
    
    

• Do you have suggestions for key contacts in these localities?
  
  

• We want to make sure we speak with the people who can best inform our landscaping and study design. Based on the nature of our questions and interests, who else would you suggest we speak with to inform the study?

  • Possible prompts:

    • Other key leaders in child care/Head Start

    • Experts on curricula and interventions, quality and quality improvement, professional development

    • Experts on CCEE data infrastructure
      
      

• (if appropriate): If a follow-up call is needed, discuss timelines and identify a point person for follow-up.

• (If appropriate): Would your office be willing to sign a letter of introduction addressed to (insert names of stakeholders as appropriate) describing the VIQI study and asking them to speak with someone from our team about the VIQI study?