CEPC Parent Survey OMB SSB_fnl clean_1.3.24

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Survey on Where Parents Look for and Find Information and How They Use Information When Selecting Child Care

OMB Information Collection Request

0970 – New Collection

Supporting Statement

Part B

JANUARY 2024

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Alysia Blandon, Bonnie Mackintosh

Part B

B1. Objectives

Study Objectives

The main objective of the study is to collect nationally representative survey data to learn about where parents look for and find information about Child Care and Early Education (CCEE); how parents assess the people, places, or things that may offer CCEE information; what types of CCEE information parents look for; and how parents use information to select CCEE. The study will expand the field’s understanding of the kinds of information parents look for and where they get information, including through official outreach and other consumer education efforts. The survey will help to identify how states and territories can reach parents to share consumer education information. We believe there is a gap in this type of information and therefore in our understanding of how consumer education can support parents as they search for and select CCEE for their young children.

Generalizability of Results

This study is intended to produce nationally representative estimates of U.S. parents and legal guardians who are at least 18 years old and with children under the age of 6 years, but not in kindergarten living in the same household. The study will include all households with children of this age group whether or not they currently use CCEE and whether or not they recently searched for CCEE.

ACF has contracted with NORC at the University of Chicago to complete this work. The study will use a nationally representative sample from NORC’s probability-based AmeriSpeak panel. For more information on this, see the sampling section below and Appendix C (pages 1-2). The study will be multi-mode and use a probability-based panel representative of the U.S. household population, including all 50 states and the District of Columbia. This ensures that American parents of various ages, race/ethnicity, education level, and gender from the full spectrum of geographic locations would be represented. To include families who may not have English as a preferred language, a Spanish version of the survey will be provided.

Appropriateness of Study Design and Methods for Planned Uses

Probability sampling results in higher quality findings because it provides an unbiased representation of the larger population of interest. It is the most rigorous design to answer our research questions, and it also allows for others to make use of the findings because it is generalizable to the larger population of parents with children under the age of 6, not yet in kindergarten. Therefore, this study design will meet the goal of gathering information on parent search and selection of CCEE that may be used by Child Care Lead Agencies to inform their consumer education efforts and inform ACF’s future planning and decision-making. The survey will also help to identify how states and territories can reach parents to share consumer education information.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

The target population for this survey includes parents and legal guardians who are at least 18 years old and have at least one child under the age of 6, but not in kindergarten who are living in the same household. Data from the 2022 March Current Population Survey indicates there were approximately 76.7 million parents/guardians who were at least 18 years of age with children under the age of 5 years living in the same household. Eligible participants do not have to currently use child care, nor do they have to currently be looking for care.

Sampling

The sampling frame will be the AmeriSpeak panel, which is representative of the U.S. household population, including all 50 states and the District of Columbia. It currently contains 48,900 panelists aged 13 and over residing in over 40,000 households. The panel is randomly selected with a known, nonzero probability from the NORC National Frame, a multistage probability sample that fully represents the U.S. household population and is supplemented with samples from USPS addresses and voter registers. The panel is recruited by mail, telephone, and field interviewers face-to-face¹. In-person recruitment of AmeriSpeak panelists enhances representativeness for young adults, lower socio-economic households, non-internet, and other households that are typically hard-to-reach for statistical surveys of the population. The current AmeriSpeak panel has a panel recruitment rate of 22 percent. This is the weighted American Association for Public Opinion Research (AAPOR) response rate at the household level for recruitment. A recruited household is a household where at least one adult successfully completed the recruitment survey and joined the panel.

As households were recruited into the AmeriSpeak panel, NORC collected a list of all individuals living in the household. In subsequent surveys, NORC collected information on parent-child relationships within the household. For the current study, households identified as having children under the age of 6 years but not in kindergarten will be sampled and given the Parent Survey Eligibility Confirmation Section AE only to confirm the existence of children under 6 years but not yet in kindergarten and of their parent(s)/legal guardians in the household. Eligible parents will continue to the Parent Survey Questionnaire Section A – DA (Instrument 1 or 1S). If a panel household has more than one eligible panel member, only one panel member is selected at random.

Given the complex design of the AmeriSpeak panel, the design effect for this survey will be approximately 2. This means that we would lose some precision on estimates compared to the simple random sampling design. We plan to collect around 1,500 completed surveys. This gives us a margin of error less than or equal to 3.6 percent for an estimate of proportion.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument

We developed the Parent Survey Eligibility Confirmation (Section AE) to confirm eligibility of AmeriSpeak panel households. This section includes questions to verify the age of respondents to ensure they are over the age of 18 as well as to identify households with children under the age of 6 years, not in kindergarten. The parent survey questionnaire is aligned to our research questions (see SSA section A2 Research Questions). To develop the parent survey questionnaire, we looked for relevant items that had been previously used in the CCEE field and other relevant contexts and revised and adapted them to fit our purpose and to address the research questions. We also created new survey items.

We conducted two rounds of cognitive testing of the English parent survey questionnaire². In the first round, we used cognitive testing to understand where parents get information from and how they think about different sources of information so that we could refine how we worded the different sources of information. We asked a variety of parents with children under the age of 6, not yet in kindergarten, who were located in different demographic cross-sections of the general parent population about where they get information about CCEE and how they think about different sources of information, how they identify sources, and what words parents use to describe different sources of information. Hearing directly from parents helped us refine the list of sources of information so that the wording in the parent survey questionnaire better reflected how parents categorized, defined, named, and thought about sources of information.

In the second round of English cognitive testing, we tested the full survey instrument with a variety of parents with children under the age of 6, not yet in kindergarten located in different demographic cross-sections of the general parent population. We used this round of cognitive testing to determine whether respondents’ understanding of the survey questions matched with researcher's intentions. This allowed our team to assess whether any survey questions or wording in items or response options caused confusion or were unclear so that we could make needed adjustments. In addition to cognitive testing, we also shared the questionnaire with the Office of Child Care (OCC) and with external experts to get their feedback. We then revised the English language instrument based on cognitive testing and feedback from OCC and experts. We will conduct a third round of cognitive testing of a Spanish version of the instrument³. The goal of this cognitive testing is to refine the Spanish version of the instrument so that the translation reflects the intentions of the researchers and creates equivalency with the English language version of the instrument.

B4. Collection of Data and Quality Control

NORC at the University of Chicago, a contractor, will collect the survey data using the AmeriSpeak panel. AmeriSpeak panel households with children under the age of 6 will be identified from AmeriSpeak’s pre-screened, nationally representative pool of participants. Sampled households will complete the Parent Survey Eligibility Confirmation Section AE only to confirm that they still meet the eligibility requirements. If eligible, panelist will immediately begin the Parent Survey Questionnaire Section A – DA (Instrument 1 or 1S).

Recruiting

We will recruit panelists using multiple contact modes (email, mail, phone, and SMS text messages where appropriate). Appendix A and Appendix A-S include recruitment materials to be used during the data collection field period. Data will be collected through an online web survey or by an AmeriSpeak interviewer over the phone. Both will be available in English and Spanish.

We plan to recruit panelists beginning with the contact modes they noted as preferred when they joined the AmeriSpeak panel.

• Panelists who have indicated that they prefer to complete surveys online will receive the initial survey invitation and reminders by email. Panelists who have consented to receive SMS messages from AmeriSpeak will also receive text reminders after the initial invitation.

• Panelists who have indicated that they prefer to complete surveys by phone will receive an initial phone call informing them of the new survey. Interviewers will attempt to complete the Parent Survey Eligibility Confirmation Section AE to confirm that they are still eligible and if yes, the Parent Survey Questionnaire Section A – DA (Instrument 1 or 1S) with them on that initial call or schedule an appointment for a future time. They will make follow-up calls to panelists who do not complete the survey on the initial call. Panelists who have consented to receive SMS messages from AmeriSpeak will also receive text reminders after the initial invitation.

• All AmeriSpeak panelists can access the Parent Survey Eligibility Confirmation Section AE and Parent Survey Questionnaire Section A – DA at any time through the AmeriSpeak member portal.

After an initial series of contacts, we will evaluate progress by reviewing data collection production metrics and determine additional contact strategies to increase response from the panelists during the remaining weeks of data collection. We will send additional recruitment materials to address barriers to participation and increase response as needed through the end of data collection. We may, for example, send a postcard mailing to remaining non-respondents coupled with a follow-up phone call or email. If we learn that certain subgroups are participating at lower rates, we may send an additional request to participate by email or mail explaining that it is important to hear from all sampled panelists.

Data collection Monitoring and Quality Assurance

To make sure we collect high quality data, we will do the following before we start to collect data:

• Questionnaire testing – We will thoroughly test the programmed questionnaire prior to the start of data collection to confirm that the instrument is functioning correctly.

• Data simulation – to confirm that responses are being recorded as expected.

• Interviewer training – The AmeriSpeak interviewers will complete a training that will teach them how to answer questions about the study, encourage panelists to complete the survey, and administer the questionnaire to individuals who prefer to complete by phone.

To monitor data collection and review the collected data, we will:

• Download and inspect the survey regularly to monitor the response rate, sample quota targets, and the level of coverage from both web and phone modes.

• Actively monitor the AmeriSpeak telephone interviewers as they begin data collection. Quality Assurance monitors will review recordings to assure that interviewers are reading verbatim, fostering a positive experience with respondents, gaining cooperation skills, probing appropriately, and entering codes and data correctly.

• Review the data and frequencies on key variables to monitor the functioning of the questionnaire and to detect any technical issues and possible mis-interpretation of questions by respondents.

• Debrief with the AmeriSpeak telephone interviewers to understand any issues that arose administering the questionnaire or questions respondents had during the interview.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The current panel retention rate is 84 percent. This is the weighted percent of recruited households that remain on the panel and are available for sampling now. Unavailable panelists are those who have temporarily or permanently asked to be removed from the panel or from receiving surveys. The panel recruitment and retention rates will be slightly different for this survey due to the panel’s yearly recruitment and changes in panelist status.

For the Parent Survey Eligibility Confirmation Section AE we are targeting a completion rate of 40 percent and a Parent Survey Questionnaire Section A – DA completion rate of 95 percent. This is based on rates obtained in other surveys conducted using the AmeriSpeak panel. In reporting our results, we will calculate nonresponse rates according to the standards recommended by AAPOR, which involve calculating:

• Parent Survey Eligibility Confirmation Section AE only completion rate = # of completed questionnaires/ # of invited to the survey

• Parent Survey Questionnaire Section A – DA completion rate = # of completed interviews/ # of eligible cases

We will also report the overall survey response rate that accounts for survey response in all phases, including panel recruitment, panel retention, Parent Survey Questionnaire (Section AE) completion, and Parent Survey Questionnaire (Section A – DA) completion⁴.

NonResponse

We will encourage participation by offering multiple modes of data collection (online and phone) and English/Spanish languages. We will use panelists’ historical survey completion rates to identify where additional outreach through mailings may be needed to obtain demographic representativeness. In addition, monitoring during data collection will allow us to identify hard-to-reach subgroups that may have lower Parent Survey Eligibility Confirmation Section AE rates or Parent Survey Questionnaire Section A - DA completion rates so that actions can be taken to encourage their response.

Unit non-response in the household/provider survey will be handled by weight adjustments using AmeriSpeak’s extensive data, such as demographics and geographic info at the household and individual levels. Where possible, item non-response will be handled via imputation. The exact method of imputation chosen will vary by data item based on prior research regarding the most appropriate method.

B6. Production of Estimates and Projections

Estimates will be produced and are intended to be generalizable to the parents of children under the age of 6, but not yet in kindergarten described in section B1. The AmeriSpeak sampling weights will be based on the panel weights developed for the AmeriSpeak panel, the selection probability from the panel into the survey, and the adjustments made for eligibility and non-response.

Because AmeriSpeak has collected rich demographic data from its panelists, these auxiliary variables could be used to form weighting adjustment cells comprising both respondents and non-respondents to correct for non-response. Use of the sampling weights will enable unbiased estimation of the descriptive statistics for the population. Strata and cluster variables will be used in conjunction with the sampling weights to produce design-corrected standard errors.

Data from this information collection will be archived and made available to the public for secondary analysis. Datasets will include sampling weights as well as strata and cluster variables to allow analysts to produce design-corrected standard errors for their analysis. Study documentation will describe how these variables can be used with commonly available statistical software to produce valid population estimates.

B7. Data Handling and Analysis

Data Handling

We will collect data primarily through a web-based survey that respondents will self-navigate. There are several steps we will take in survey design, programming, and then data processing, to minimize errors in data collection. We will design our questionnaire with minimal skip patterns or other features that may require high cognitive demand for the respondent. Questionnaire program strategies will also help by providing ranges for numeric items, presenting numbers entered in words, and not allowing for invalid codes for fixed-coded items.

We will conduct regularly scheduled reviews of statistical programs and data files during the data collection and analysis process. These reviews will check for errors due to data entry, coding, and processing. This includes, for example, inconsistencies in open-ended coding, errors in variable creation (for example, creating variables from skip patterns or grids), and errors in analysis (for example, using survey weights or statistical tests incorrectly).

After fielding the Parent Survey Eligibility Confirmation Section AE and Parent Survey Questionnaire Section A – DA and during the data processing period, we will code open-ended items and create variables. Coding of verbatim responses will undergo 100 percent double blind coding with reconciliation, and error rates exceeding 3 percent trigger a second round of more expert coding.

Variable creation (e.g., combining response or question items to create a single construct) and analysis will also have strict quality assurance protocols with an initial review of planned procedures for variable creation, review of statistical programming code, and then verification of analytic output (including against comparison data when available).

Data Analysis

The analytic focus of the study is to document and identify patterns in parents’ knowledge and use of different sources of information during their CCEE search and selection process. We will also document and explore parent perspectives and experiences with using different information sources on CCEE, including the challenges parents face when they try to find information about CCEE. We will begin analysis by creating key variables of interest (individual survey items and/or factors) and inspecting their distributions and rates of missingness. If contextual information from other data sources is required (e.g., about local characteristics or information about state consumer-education resources), we will prepare those variables during data collection to have available for analysis. For example, we may use American Community Survey (ACS) or similar external data sources to better understand the communities where households live. Because the main analytic objective is descriptive in nature, statistics will be focused on weighted frequencies, means, and proportions.

We may progress from univariate data inspection (frequencies, means, and proportions) to bivariate comparison (e.g., t-tests and chi-squared tests of means and frequencies across categories of interest, such as child age) and possibly multivariate modeling (including for example regression, latent class analysis), depending on analytic capacity, time, and guidance in collaboration with OPRE. We will prioritize examining survey items by household demographic information that is of most interest (for example, child age). We will work to identify the contextual variables that will best meet the research questions. We will share early tabulations and findings to identify themes and insights with OPRE. In accordance with the ACF Evaluation Policy’s principle of Transparency, NORC will pre-register the study with the ClinicalTrials.gov, and a full analysis plan for the study will be publicly posted on this site prior to the start of data analysis.

Data Use

We will share findings through both practitioner and researcher-oriented dissemination activities. Our findings will fill a research gap in the field’s understanding of how parents search for and select care for their young children. We will produce published reports to share with Child Care and Development Fund (CCDF) lead agencies, technical assistance centers that support CCDF lead agencies on consumer education, and the broader child care and early education field. This information may support peer learning and innovation to advance the field. ACF will also use this information internally to inform future planning.

Data, questionnaire, and methodology files will be prepared and submitted to be archived in an online data repository for public use. The archived data will include thorough documentation of data collection and cleaning procedures, and will be available so that a wide variety of researchers, practitioners, and interested individuals with lived experience are able to access, use, and duplicate analyses conducted through the project.

B8. Contact Persons

Name	Affiliation	Email Address
Alysia Blandon	OPRE/ACF	Alysia.Blandon@acf.hhs.gov
Bonnie Mackintosh	OPRE/ACF	bonnie.mackintosh@acf.hhs.gov
Kanru Xia	NORC at the University of Chicago	Xia-Kanru@norc.org
Sarah Kabourek	NORC at the University of Chicago	Kabourek-Sarah@norc.org

Attachments

Instrument 1. Parent Survey Eligibility Confirmation (Section AE) and Parent Survey Questionnaire (Section A - DA)

Instrument 1S. Spanish Translation of Instrument 1 Parent Survey Eligibility Confirmation (Section AE) and Parent Survey Questionnaire (Section A - DA)

Appendix A. Recruitment and Communication Materials English

Appendix AS. Recruitment and Communication Materials Spanish

Appendix B. References

Appendix C. Technical Overview for the AmeriSpeak panel

1 The technical overview of the AmeriSpeak panel can be found in Appendix C.

2 Cognitive testing was completed under the Fast Track Generic Clearance for Collection of Qualitative Feedback on Agency Service Delivery (OMB #: 0970-0401)

3 Cognitive testing is being completed under the Fast Track Generic Clearance for Collection of Qualitative Feedback on Agency Service Delivery (OMB #: 0970-0401). Once complete, if changes are needed, we will submit the final version through a nonsubstantive change request.

4 References for AmeriSpeak panel quality can be found in Appendix C.

8