SSB_VIQI Formative Generic

Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education

Generic OMB Information Collection Request for Formative Data Collection

0970-0356

Supporting Statement Part B

February 2017

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Ivelisse Martinez-Beck

The purpose of the current Information Collection Request (ICR) is to gather preliminary information via tailored, semi-structured discussions from national, regional, state, and local informants about the child care and early education (CCEE) program and policy context to inform the research design, recruitment, and sampling strategies used for the pilot and full-scale phases of a research and evaluation study sponsored by the Office of Planning, Research and Evaluation (OPRE) called Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education.

B1. Respondent Universe and Sampling Methods

This ICR for the VIQI project seeks to gather information from key informants in the field of CCEE. First, the study team will develop a list of possible informants based on recommendations from the Contracting Officer’s Representative (COR) and other staff in the Office of Planning, Research and Evaluation (OPRE), referrals from other informants, and Internet searches related to the CCEE topics listed below.

Informants in this Information Collection (IC) will include:

• National and regional stakeholders such as agency staff connected to CCEE programs, researchers, practitioners, and leaders in the CCEE field including experts in CCEE policy, early childhood interventions, curricula, implementation of quality-enhancing initiatives, contextual influences, professional development of the early childhood workforce, and quality of CCEE programs; and

• State and local stakeholders such as CCEE early childhood program administrators, local leaders in early childhood education, and local CCEE practitioners.

Next, the study team will use a purposeful, snowball sampling strategy to determine which informants to engage and when. The informants will be selected to participate in an iterative fashion based on their expertise and the study’s need for information from different sources and localities. The team will engage informants in waves, consistently returning to the list of potential informants to determine which individuals with select expertise should be engaged next given the study team’s remaining gaps in knowledge. An alternative to this sampling strategy would be to seek a representative sample, but obtaining a probability-based sample to reach the desired subpopulations of interest for this IC would be cost-prohibitive and not essential for achieving study goals.

Participation in this IC is voluntary. Based on prior efforts to gather insights and information from key informants in the early design phases of studies, we have typically found that informants are eager to participate in these types of discussions. We therefore expect a fairly high response rate (80% participation rate) among those informants who are contacted and invited to participate.

We anticipate meeting with up to 100 state and local informants and 40 national and regional informants over the period covered under this IC (approximately 13 months from OMB approval).

B2. Procedures for Collection of Information

Each facilitator team composed of at least one senior and one junior researcher will make initial e-mail contacts, secure informant participation, and conduct the tailored, semi-structured phone or in-person discussions. The senior researchers are experienced in the process of gathering information for purposes of informing the early design of pilot, demonstration, and evaluation studies, such as VIQI. In addition, all involved in this data collection will receive training to ensure that informants are engaged in a consistent manner.

The remainder of this section describes the facilitator teams’ procedures for contacting informants.

Informants will be selected and contacted in waves, so that the facilitator teams can use the information obtained in previous waves to refine the selection of subsequent informants on an ongoing basis. We envision phone discussions with national and regional CCEE informants and a combination of phone and in-person discussions with state and local informants. Prior to any in-person visits, the decision to conduct the discussion in person instead of by phone will be made by the study team in mutual agreement with OPRE.

In all waves, the facilitator teams will:

• Send informants an e-mail invitation to participate in the discussion (see Attachment C: VIQI Phone Discussion Email Templates). The email communication will introduce the study and its goals and the facilitator team and will offer suggested times for the discussion. The email will also state that participation in the discussion is voluntary.

• Send informants the project description (see Attachment D: VIQI Project Description) and an agenda to guide the discussion (see Attachment E: VIQI Landscaping Discussions Sample Agenda).

• Seek to involve multiple (approximately 2-3) informants in discussions where possible and appropriate, rather than conducting only one-on-one meetings. This strategy leverages efficient communication strategies.

• Lead the discussion using a subset of the most relevant questions from the semi-structured protocol based on each informant’s expertise and our current gaps in knowledge (see Attachment A: Semi-Structured Protocol for VIQI Landscaping Discussions (National/Regional Version) and Attachment B: Semi-Structured Protocol for VIQI Landscaping Discussions (State/Local Version)).

• Ensure that the discussion and any follow-up discussion use no more than a total of 1.5 hours of each informant’s time.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

We expect little nonresponse. Using the study team’s past experience with engaging similar national, regional, state, and local informants to collect information for similar purposes, the team expects approximately 80% participation. The purpose of this IC is to inform the research design, recruitment, and sampling strategies used for the pilot and full-scale phases of VIQI. Thus, we plan on using the information gathered to refine study priorities and research questions, guide the study’s framing and study design, anticipate challenges, and identify potential solutions. We anticipate that the vast majority of informants will likely be interested in providing their expertise to inform a national study.

Dealing with Nonresponse

If a potential informant does not respond to our email request for a meeting, we will send a second request by e-mail. If we do not receive a response to our second e-mail, the team may follow up with a phone call. After three attempts, the facilitators will not continue to pursue a response.

Maximizing Response Rates

We will take several steps to help ensure a high rate of participation among informants including the following:

• The informants will be contacted by an assigned facilitator team, which will include one senior and one junior staff member from the study team. The senior members have significant experience in working closely with programs and their stakeholders on previous evaluations and have had a high level of success in engaging individuals in preliminary discussions such as these.

• Our initial e-mail request for participation will emphasize the importance of participation and how the study will benefit the CCEE field.

• We will leverage existing relationships to help secure buy-in from potential informants. For example, we will ask OPRE, the Office of Child Care, and the Office of Head Start for recommendations of and introductions to informants based on the needs of the study.

• As we engage additional informants, we may also ask for their recommendations of other national, regional, state, and local informants based on the current needs of the study.

• The facilitator teams will be accommodating of respondents’ schedules. We will be flexible about the meeting dates and times.

B4. Tests of Procedures or Methods to be Undertaken

The data collection instruments will not be pre-tested. Previous large-scale evaluations have successfully used similar semi-structured discussion protocols during information gathering processes.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Ivelisse Martinez-Beck
Team Leader for Child Care Research/Senior Social Science Research Analyst, OPRE

Amy Madigan
Team Leader for Head Start Research/Senior Social Science Research Analyst, OPRE

Tracy Carter Clopêt‎
Social Science Research Analyst/Contractor, OPRE

4