Supporting Statement B - NSECE COVID Followup

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

2019 National Survey of Early Care and Education COVID-19 Follow-up

OMB Information Collection Request

0970 - 0391

Supporting Statement

Part B

October 2020

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Ivelisse Martinez-Beck and Ann Rivera

Part B

B1. Objectives

Study Objectives

The objective of the National Survey of Early Care and Education (NSECE) is to document the nation’s current supply of early care and education (ECE) services. Because the COVID-19 pandemic is likely to have had an impact on the ECE supply, and because the availability of ECE services is critical to restoring the U.S. economy, updated data on the ECE supply will be crucial for policy-making and research. The purpose of the NSECE COVID-19 Follow-up study is to describe the impact of the COVID-19 pandemic on the pre-pandemic ECE supply and the ECE workforce, including changes in supply or departures from and re-entries to the workforce. Data from the study can be used to estimate the impact of the pandemic on ECE supply and the financial health of ECE providers, as well as providers’ experiences during and after the pandemic emergency. Through a two-wave design, the NSECE COVID-19 Follow-up data will trace providers’ and workforce members’ experiences from 2019, through the initial months of the pandemic (to be reported in the Wave 1 interview), and then to the one-year point since the onset of the pandemic (to be reported in the Wave 2 interview). This design is intended to capture the trajectories of providers and workforce members, for example through recovery back to full or modified participation in ECE supply, or long-term exits from ECE supply. Prospectively collecting these data allows inference about the types of providers and workforce members most likely to have exited or survived in ECE, as well as the programmatic supports received by those who remain in ECE at the time of the second interview.

Generalizability of Results

The NSECE COVID-19 Follow-up study is intended to produce nationally-representative estimates of the characteristics of the following groups of providers:

• individuals who provided paid care for children under the age of 13 in a residential setting as of 2019 (Home-based Providers);

• providers of care to children ages 0 through 5 years of age (not yet in kindergarten) in a non-residential setting as of 2019 (Center-based Providers); and

• individuals employed in center-based child care programs working directly with children in classrooms as of 2019 (Center-based Classroom Staff).

This study is not intended to produce estimates of the overall ECE supply as of 2020 or 2021.

Appropriateness of Study Design and Methods for Planned Uses

The 2019 NSECE used administrative lists of ECE providers and a commercially available list of schools and ECE providers to develop the provider sampling frame, from which probability samples of Home-based and Center-based Providers were drawn. The 2019 NSECE also used its Household survey to identify individuals providing ECE services but who were not present in the provider sampling frame. These individuals were also administered the provider survey. The 2019 NSECE was able to collect data from nationally representative samples totalling about 16,000 across three types of ECE providers and workforce members who were providing ECE services as of January - July 2019.

The goal of the NSECE COVID-19 Follow-up study is to describe the impact of COVID-19 on the 2019 ECE supply. To do so, the Follow-up study will re-interview providers that completed interviews in the 2019 study (with some exceptions: see section B2), regardless of whether or not they are still in the ECE sector at the time of the follow-up surveys. It is also important to note that the Follow-up study does not include providers who entered the ECE sector after the summer of 2019. Therefore, this study is not intended to describe the overall ECE supply as of 2020 or 2021. This and other limitations of the Follow-up study will be included in written products associated with the study.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

The 2019 NSECE study collected data on three distinct populations, as described in Section B1. Based on figures from the 2012 NSECE, estimated population sizes are 1 million individuals for paid Home-based Providers; 129,000 programs for Center-based Providers; and about 1 million individuals for Center-based Classroom Staff. In addition, the NSECE distinguishes between two types of paid Home-based Providers: (i) listed Home-based Providers, who are licensed, license-exempt, registered, regulated, and otherwise listed home-based ECE providers identified from state or national administrative lists, and (ii) unlisted Home-based Providers, who do not appear on any administrative lists and are identified from the household screener in the NSECE survey. Of the estimated 1 million paid Home-based Providers, about 120,000 were listed Home-based Providers, and about 920,000 were unlisted Providers.

The Follow-up study aims to collect data from programs and individuals that completed interviews in the 2019 study, regardless of whether or not they are still providing ECE services at the time of the follow-up surveys, with some exceptions (see Sampling subsection below). The study has a two-wave design, with surveys to be fielded in Fall/Winter 2020 and Spring 2021. Wave 1 of the study includes all providers that completed interviews in 2019. For Home-based Providers and Center-based Classroom Staff, Wave 1 of the study will collect data from the same individuals interviewed in 2019. For Center-based Providers, Wave 1 of the study will collect data from center directors interviewed in 2019 or, if they are unavailable, other individuals knowledgeable about the center’s operations. Wave 1 of the study will collect data on whether providers were still providing ECE services in February 2020, that is, prior to the pandemic emergency. Providers or workforce members that had permanently ceased ECE services prior to February 2020 would not be fielded for Wave 2. Any providers or workforce members who were participating in ECE provision at the onset of the pandemic will be fielded for Wave 2, so that trajectories of ECE participation, including temporary closures as well as permanent exits, can be identified in the Wave 2 interview. We note that home-based provider survey respondents are both ECE providers as well as members of the ECE workforce.

Sampling

The Follow-up study will re-interview programs and individuals that completed interviews in the 2019 NSECE study. Listed Home-based Provider and Center-based Provider respondents were included in a sample of providers originally drawn in 2018 using a multi-stage sampling design. Unlisted Home-based Providers were identified from a sample of households, also drawn using a multi-stage sampling design. Center-based classroom staff were randomly selected from staff members belonging to randomly selected classrooms at a Center-based Provider respondent. The survey designs for the 2019 study are described in the supporting materials for the information collection under OMB # 0970-0391.

The number of programs and individuals that the study will attempt to contact are approximately:

• Home-based Providers: 4,600 (of which 4,200 are Listed, and 400 are paid unlisted Home-based providers)

• Center-based Providers: 6,100

• Center-based Classroom Staff: 4,500

The 2019 NSECE also collected data from an additional 800 Center-based Providers and 700 Center-based Classroom Staff that will not be re-contacted for the Follow-up study. Institutional approvals are required to interview these providers. Because the Follow-up study is prioritizing timely data collection and dissemination in order to better understand and address the impact of COVID-19 on ECE supply, the abbreviated timeline of the study precludes obtaining these approvals. Administrative data, where available, will be used to supplement primary data collection.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument(s)

The vast majority of items proposed for the three questionnaires come from other studies that have successfully fielded the items to understand ECE supply or with comparable populations, especially including the 2019 and 2012 NSECE instruments themselves. In addition, we received feedback on the submitted questionnaires from several individuals who have conducted surveys with ECE provider populations (regarding the appropriateness of questionnaire design for collecting high quality data), as well as from ACF program staff (regarding the relevance of questionnaire design to achieve the agency’s goals). We also conducted cognitive interviews with members of the target population to understand issues such as ability to recall requested information or to populate codeframes within the questionnaire.

The design takes a multi-layered approach to identifying and minimizing measurement error, beginning with using well-tested questions with high-quality benchmarking data wherever possible. This ensures that questions are well-functioning and that collected data can be compared with other sources, including the 2012 and 2019 NSECE data, as well as the 2020 Household Pulse surveys from the U.S. Census Bureau. Questionnaire functioning will be monitored during data collection for evidence of respondent mis-comprehension, such as misplaced decimals, high incidence of break-offs at individual items, or unusual distributions on key variables. We will also monitor data collection for evidence of systematic patterns of non-response, and for internal consistency of responses.

B4. Collection of Data and Quality Control

NORC at the University of Chicago, a contractor, will be collecting data from the center-based providers, home-based providers (listed and unlisted paid),and classroom staff who participated in the 2019 NSECE for both waves of the NSECE 2019 COVID-19 Follow-up Study. Below we describe the proposed contacting approach which is similar to the one we employed successfully in the 2012 and 2019 NSECE data collection efforts. While we will not be visiting provider locations in person because of the pandemic, we managed to complete a number of cases in 2012 and 2019 through phone, mail, and email outreach alone. We anticipate that the contact information collected in 2019 and the recognition of the study name will help compensate to some extent for our inability to make in-person visits.

The three study questionnaires will be programmed for completion online by the respondent or by phone with an interviewer. The home-based and workforce questionnaires will be available in English or Spanish. Providers and classroom staff will be contacted first by mail and email (when available). Because eligibility for the study has already been established through the 2019 NSECE, our contact materials and approaches can focus on motivating the sample to participate from the outset.

The initial outreach series will consist of three contacts: a survey invitation letter or email, a thank you/reminder postcard sent a week later, and a follow-up letter, mailed two weeks after the postcard. These contacts will invite providers and classroom staff to complete the questionnaire online. After the initial contact, field interviewers will follow up with providers and classroom staff by phone. The interviewers will answer questions about the study, encourage people to complete the follow-up survey, and provide any technical assistance that respondents may need to access and complete the online questionnaire. Interviewers will also be able to complete the interview by phone if needed. For providers and classroom staff who do not have a good mailing address, interviewers will make contact by phone first to introduce the follow-up study and attempt to collect new contact information to send study materials. Additional mailings and emails may be sent during data collection to address common reasons for refusal or other barriers to participation.

A series of consistency and range checks will be built into the questionnaires to prevent invalid responses from being recorded. The data collection team will thoroughly test programmed questionnaires prior to the start of data collection. This testing will include a review of the data to confirm that responses are being recorded as expected. Throughout data collection, we monitor the functioning of the questionnaire to detect potential technical issues and possible mis-interpretation of questions by respondents. We will produce regular data quality assurance reports that collate questionnaire data across variables of high analytic value to help identify such issues and allow us to take corrective action.

In addition to questionnaire functioning, NORC monitors data collection progress carefully throughout the field period to achieve good response rates and representative data. Daily production reports will show how data collection is progressing across each sample and enable us to identify problem areas and take remedial action quickly when needed. These reports will also allow us to monitor completion rates by sample subgroups in order to protect against bias in the sample. When we detect that a subgroup is completing surveys at lower rates, we will adjust our field procedures to boost completion among that group.

To monitor field interviewer performance specifically, the NSECE team will develop a series of performance metrics built from multiple data sources (e.g., item non-response rates, questionnaire completion rates, case prompting rates). These metrics, produced weekly, show where interviewers are deviating from the norm on key performance measures, thereby revealing areas where re-training or special coaching may be needed.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

We are targeting COVID follow-up interview completion rates of 75 percent for each sample type and have presented burden estimates assuming 75 percent of the eligible sample participates in each wave of data collection, with some sample members participating in only wave 1 or only wave 2, but the majority of completed interviews corresponding to sample members completing both waves. Ensuring that the achieved sample is representative based on 2019 characteristics will be even more important than interview completion rates in ensuring that representative estimates can be generated from collected data. Since our goal is to interview sample members participating in the 2019 NSECE, we do not propose to include as stages of response the participation rates in the 2019 NSECE.

Wave I interview completion rate = # of completed interviews

(# of 2019 completed interviews)

Wave II interview completion rate = # of completed interviews

(# of 2019 completed interviews - # of W1 cases having exited ECE prior to 2/20)

We note that we are not proposing to interview center-based providers who required research review as part of the 2019 NSECE fielding. We will seek to collect administrative data on these providers where possible, but exclude them from the center-based completion rate and will treat them separately in analyses.

NonResponse

Because the COVID Follow-up will involve a longitudinal sample, extensive data will be available from the 2019 NSECE to assess and potentially correct for non-response to the COVID Follow-up. Sampling weights incorporating non-response adjustments will allow estimation of the full population from 2019. Monitoring during data collection will allow identification of subgroups of interest that may be lagging in cooperation so that remedial actions can be taken to encourage their response. If these are not successful, and if non-response adjustments during data collection are not adequate, then multiple imputation may be undertaken for key variables. All eligible individuals will be fielded for both Wave 1 and Wave 2, regardless of participation in Wave 1, allowing some Wave 1 non-respondents to return in Wave 2.

We have devised a data collection approach that does not require in-person outreach, given the uncertainty of the current pandemic and associated social distancing requirements. We are expecting that some center-based providers may be closed due to the pandemic, and therefore it may be more difficult to locate and gain cooperation from respondents representing those centers. We also anticipate that some workforce members or home-based providers may have discontinued internet or cell-phone service due to financial difficulties, which may diminish our ability to reach them using contact information that was valid in spring of 2019. In contrast, the strong salience of being able to report the impact of the pandemic on providers’ and workforce members’ personal lives and ability to provide ECE may be a strong motivator that compensates at least somewhat for increased operational challenges in fielding the data collection.

Questionnaires are designed to minimize item-non response based on design work completed prior to the 2012 NSECE as well as further work for the 2019 NSECE. For example, financial questions often have a follow-up for those who decline, asking them to report ranges rather than exact values. These methods significantly reduced item non-response for selected items in the 2019 NSECE relative to the 2012 NSECE, although overall item non-response in both years was very low.

B6. Production of Estimates and Projections

Estimates will be produced for official external release by OPRE, and are intended to be generalizable to the three groups of ECE providers described in section B1. Sampling weights for the Follow-up study will be based on the weights developed for the 2019 NSECE, with adjustments made for non-response in each of the two waves. Because the Follow-up study re-interviews respondents from the 2019 NSECE, it will be able to draw on a rich set of provider characteristics collected in the 2019 study – in addition to design variables such as strata and clusters – to correct for panel attrition. These auxiliary variables will be used to form weighting adjustment cells comprising both respondents and non-respondents. The weighting adjustment is then the inverse of the response rate in each cell. Use of the sampling weights will enable unbiased estimation of the descriptive statistics for each of the three population of providers. Strata and cluster variables will be used in conjunction with the sampling weights to produce design-corrected standard errors.

As with the 2019 NSECE, selected data from the information collection will be made available to the public for secondary analysis. Datasets will include sampling weights as well as strata and cluster variables to allow analysts to produce design-corrected standard errors for their analysis. Study documentation will describe how these variables can be used with commonly available statistical software to produce valid population estimates.

B7. Data Handling and Analysis

Data Handling

Procedures to minimize errors in the data begin with designing a questionnaire which will collect accurate data and which has skip patterns that minimize situations where respondents are unable to navigate appropriately. Questionnaire programming builds on this through techniques such as ranges for numeric items, presenting in words any numbers entered, and the inability of data entry of invalid codes for fixed-coded items. Coding of verbatim responses undergoes 100 percent double blind coding with reconciliation, and error rates exceeding 3 percent trigger a second round of more expert coding. Variable creation and analysis also have strict quality assurance protocols with review at a specifications development stage, review of statistical programming code, and then verification of analytic output (including against comparison data when available).

The NSECE data files are developed with a strong commitment to transparency. Raw data, prior to any edits or imputation are always available to interested data users, potentially through restricted use mechanisms provided by the agency. If editing or imputation are elected for specific variables, the contributions of those edits or imputations are easily visible to data users who might wish to critique or replicate the edits. We will not impute raw data (such as reports of income) but may impute created variables (such as estimated HH income to poverty ratio) for completeness.

Data Analysis

An analytic focus of the Follow-up study is to document the impact of COVID-19 on 2019 NSECE providers that were still active in February 2020, i.e. prior to the pandemic emergency. A number of analyses will thus compare statistics generated from the same survey items across three sets of data: the 2019 NSECE, Wave 1 of the Follow-up study, and Wave 2 of the Follow-up study. Such items may describe program characteristics, such as whether a program was in operation or the number and ages of children served, or provider characteristics, such as providers’ income or insurance coverage. In order to validate the use of 2019 NSECE data as a baseline against which data from the Follow-up study can be compared, the study will collect data on whether 2019 respondents were still providing ECE in February 2020, and if so, their total enrollment of children age 5-years and under, not in kindergarten. If a large proportion of providers from 2019 were still active in February 2020, then the 2019 NSECE can be used as a baseline for comparison against estimates from the two Follow-up waves.

In addition, the Follow-up study will introduce new items to track the financial, administrative, and operational experiences of providers through the course of the pandemic. Because the main analytic objective is descriptive in nature, statistics will be focused on weighted frequencies, means, and proportions.

Data Use

The NSECE has established a deep library of tools for data users and others to understand how to properly interpret, analyze and evaluate information from the collection. These include user guides, tutorials on how best to use the data, and analytic products that describe key findings. All materials will be posted on the Child and Family Data Archive funded by ACF, in conjunction with existing NSECE materials.

B8. Contact Person(s)

Kirk Wolter, Principal Statistical Advisor wolter-kirk@norc.org can answer questions about statistical aspects of the survey. A Rupa Datta, Senior Fellow datta-rupa@norc.uchicago.edu will supervise the collection, processing and analysis of data for OPRE. Wolter and Datta both work at NORC at the University of Chicago, the contractor for the NSECE COVID-19 Follow-up.

Attachments

Instrument 1: NSECE COVID-19 Follow-up Home-based Provider Questionnaire

Instrument 2: NSECE COVID-19 Follow-up Center-based Provider Questionnaire

Instrument 3: NSECE COVID-19 Follow-up Workforce Questionnaire

Appendix A: NSECE COVID-19 Follow-up Study Research Questions

Appendix B: Center-based Provider COVID-19 Follow-up Survey Contact Materials

Appendix C: Home-based Provider COVID-19 Follow-up Survey Contact Materials

Appendix D: Workforce COVID-19 Follow-up Survey Contact Materials

7