SSB - Formative GenIC Research - TRLECE

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

The Role of Licensing in Early Care and Education (TRLECE)

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part B

AUGUST 2022

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Ivelisse Martinez Beck, Ph.D.

Part B

B1. Objectives

Study Objectives

The proposed survey has one primary objective: to gather information from licensing administrators regarding front-line child care licensing staff members’ work schedules and contact information, to prepare for a national survey of front-line child care licensing staff. Findings from this data collection will inform a future front-line licensing staff survey that is part of the larger The Role of Licensing in Early Care and Education (TRLECE) project and will be covered under a future full OMB package.

Generalizability of Results

This data collection is intended to prepare for a national survey of front-line child care licensing staff, not to promote statistical generalization to other populations.

Appropriateness of Study Design and Methods for Planned Uses

We will use an online survey to collect information directly from licensing administrators. We will use closed-ended questions to collect information on topics where the range of likely responses is known. If the respondent cannot provide the needed information, we will contact them to discuss other ways the survey can include their state. This approach will minimize burden while also providing sufficient detail to answer the proposed guiding questions. The information gathered will be purely descriptive. We will not collect evaluation information and the information collected will not be used to assess state systems or evaluate impact.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.  

B2. Methods and Design

Target Population

The target population for this study includes the child care licensing administrators in each U.S. state and the District of Columbia (N=51).

Sampling

We are proposing to include all child care licensing administrators in our survey rather than selecting a sample. As explained in Supporting Statement A1, the front-line licensing staff population has not been studied in the past, so we know very little about it. As such, we propose collecting information about front-line staff from each state and the District of Columbia (N=51). As the entire population of child care licensing administrators is relatively small (N=51), including and following up with all of them is feasible and will likely result in our being able to include all states and the District of Columbia in the future survey of front-line licensing staff.

As described above, we plan to use a census data collection and will invite all child care licensing administrators in all U.S. states and the District of Columbia to participate.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument

The licensing administrator survey to prepare for front-line staff data collection is designed to collect work schedules and contact information for front-line child care licensing staff. The instrument was developed by a team of researchers based on project team expertise. The instrument was reviewed by the internal project team, OPRE project officers, the TRLECE Technical Expert Panel (TEP), and a separate panel of state licensing staff to ensure only questions necessary for achieving data collection objectives are asked.

B4. Collection of Data and Quality Control

Who will be collecting the data (e.g., agency, contractor, local health departments)?

Project team members from the contractor (Child Trends) will create the online survey and manage survey distribution and ongoing monitoring of survey responses. After two email attempts, a member of the TRLECE leadership (task lead, activity lead, PI, PD, or similar) will call Child Care and Early Education (CCEE) licensing administrators to gather the information and/or troubleshoot any challenges associated with sharing the needed information (e.g., need for a data sharing agreement).

What is the recruitment protocol?

Table B4 summarizes our outreach plan for CCEE licensing administrators. We will first contact all licensing administrators in all U.S. states and the District of Columbia via email. The email will describe the study and ask them to respond to the survey. The link to the licensing administrator survey to prepare for front-line staff data collection (Instrument 1) will be included in the email. Licensing administrators will be able to upload or type out a list of their front-line licensing staff members and associated work email addresses and work telephone numbers.

During outreach we will encourage licensing administrators to call us with any concerns. If licensing administrators are hesitant to provide the needed contact information, we will work with them individually to address their concerns and find flexible solutions for reaching their front-line staff. For instance, some licensing administrators may need permission from someone else in their agency before sending us staff contact information. In that case, we will provide whatever written materials they request, or we will speak directly to the person whose permission is needed to explain our goals and the request. If we are unable to address the state’s concerns, we will work with the licensing administrator to have them send an anonymous survey link directly to their front-line licensing staff, without sharing contact information with us. We will reach out to licensing administrators multiple times using various modalities to maximize the response rates. We will start by emailing all licensing administrators to explain the project and request their help. See Appendix B for examples of the initial email and reminders we will send. For those that do not respond or who request additional information, we will telephone them to provide answers to their questions and troubleshoot any challenges they may face in sharing the information with our project. Appendix B includes telephone scripts and question/answer examples.

What is the mode of data collection?

Most data collection will be conducted via an online survey hosted through REDCap, our secure online data collection platform. In cases where the survey is administered over the phone (during the third and fifth outreach attempts or they elect to call us), responses will be entered into REDCap by the member of the project leadership who makes/receives the call.

What data evaluation activities are planned as part of monitoring for quality and consistency in this collection, such as re-interviews?

REDCap will generate a unique survey link for the child care licensing administrator in each state and the District of Columbia. We will conduct several rounds of testing to ensure the survey (e.g., skip logic, survey piping) and links are working properly before sending the survey links to each licensing administrator. While respondents are taking the survey, REDCap’s built-in validation functions will ensure responses are within expected ranges and will confirm that questions that have been left blank were intentionally skipped. If the response does not pass validation, the participant will be prompted to correct the response. If participants start the survey but do not complete the survey, reminder emails will be sent as part of the outreach efforts.

Throughout the data collection, we will monitor the survey responses weekly and conduct weekly quality assurance checks on the data.

What is the proposed approach for selecting, recruiting, and training data collectors?

The task lead and activity lead will make the majority of the phone calls (outreach attempt #3 and #5) because we anticipate that each will require problem solving. If other individuals are needed to make phone calls due to time constraints, the task lead and activity lead will train and closely supervise those individuals to ensure standardized data collection.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

We will maximize response rates by providing a compelling rationale for how completing the survey will help the field, and sending reminders using multiple modalities (i.e., email, phone, text message, as possible).

This is a one-time census survey in which all licensing administrators will be asked to participate (N = 51). We anticipate that all states will respond, even if it is only to inform us that they cannot provide the needed information. We think all will respond because we recently conducted interviews with this same population (OMB# 0970-0356; Understanding Child Care Licensing Challenges, Needs, and Use of Data) and 48 out of 56 responded to those interviews. That was a much more time-intensive request, with interviews lasting up to 75 minutes, and we did not offer a token of appreciation. We believe that this much shorter request will result in a 100% response.

If we do not have any response from some states, we will calculate the unit-level response rate by dividing the number of responses by 51.

Respondents will be permitted to skip any questions they do not want to answer. Item-level response rates will be calculated for each item. To calculate item level response rates, we will divide the number of valid surveys where the question was presented by the number of valid answers to each item. Item level response rates will be reported in technical documentation, when necessary to explain analysis decisions or contextualize results.

NonResponse

Participant NonResponse

Participants will not be randomly sampled, and we will not be aggregating the information we collect across respondents, so non-response bias will not be calculated.

Item NonResponse

Respondents will be permitted to skip any questions they do not want to answer. The information will be used solely to inform future data collection, so we will not aggregate the information across respondents. For this reason, if respondents skip items we will simply not be able to use that information. No imputation will be conducted.

B6. Production of Estimates and Projections

The data will not be used to generate population estimates, either for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

The study team will build validation checks into the REDCap survey platform to ensure responses are within expected ranges and questions are not inadvertently left blank. Skip logic will allow respondents to only respond to questions that are relevant to them. The data will be stored on REDCap’s secure server. Only research team members who have completed human subjects research and data security training will have access to data collected through the REDCap survey.

Data Disposition Plan

Once data collection is complete, data will be downloaded to Child Trends’ secure server. Because there is only one licensing administrator per state, information in the data set about participants’ state will make the data identifiable. State information will be stored separately from the survey data. An ID variable will be used to connect state information with each administrator response and the linking information will be stored in a password-protected file on Child Trends’ secure server. Front-line staff contact information will remain stored on REDCap. Names and contact information will be retained until the end of the project in REDCap. Information regarding participants’ state will be retained for 3 years following the end of the project and then permanently deleted. 

Data Analysis

The data collected under this generic clearance will not be aggregated or analyzed. Instead, they will be used to inform our upcoming survey of front-line staff, which will be part of a future full OMB package.

Data Use

The results from the survey data will be used to inform a future front-line child care licensing staff survey that is part of the larger TRLECE project. The front-line child care licensing staff survey will provide insight into front-line staff’s demographic characteristics, roles, responsibilities, and perceptions of the licensing system. This will provide important information about the CCEE licensing system to policymakers, state administrators, and those working in licensing systems themselves.

B8. Contact Persons

Kelly Maxwell

TRLECE PI

Senior Research Scholar for Early Childhood Department

Child Trends

1516 E Franklin St, Suite 205 | Chapel Hill, NC 27514

kmaxwell@childtrends.org

(919) 869-3251

Diane Early

Research Scholar for Early Childhood Department

Child Trends

7315 Wisconsin Ave, Suite 1200W | Bethesda, MD 20814

dearly@childtrends.org

(406) 507-2037

Ivelisse Martinez Beck

Child Care Research Team Lead, Project Officer

Office of Planning, Research, and Evaluation

Administration for Children and Families

US Department of Health and Human Services

330 C Street, SW | Washington, DC 20024

imartinezbeck@acf.hhs.gov

(202) 690-7885

Attachments

Instrument 1: Licensing administrator survey

Appendix A: IRB exemption letter
Appendix B: Recruitment of licensing administrators
Appendix C: Project flyer for licensing administrators
Appendix D: Letter of support from OCC

7