Early Childhood Training and Technical Assistance (T/TA) Cross-System
Evaluation Project
OMB Information Collection Request - Formative Data Collection for Informing Policy Research
0970-0356
Supporting Statement
Part A
May 2016
Submitted By:
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
330 C Street, SW
4th Floor
Washington, DC 20201
Project Officer:
Ann Rivera, Ph.D.
A1. Necessity for the Data Collection
The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) requests permission to contact key stakeholders of ACF’s Early Childhood Training and Technical Assistance (EC T/TA) System for the purpose of gathering preliminary information to inform the design of a future evaluation project and gather feedback on a conceptual framework of the T/TA system. The study team will collect information for developing evaluation design options, resources, and strategies to support evaluation activities among T/TA system stakeholders. The information collected will be used for internal purposes only. This proposed information collection is requested under ACF’s generic clearance for Formative Data Collection for Informing Policy Research (OMB Number 0970-0356).
The Administration for Children and Families has launched the Early Childhood Training and Technical Assistance (EC T/TA) Cross-System Evaluation Project. Operating on national, regional, and state levels, the federal EC T/TA system supports high quality services for children and families. To inform the continuous improvement of ACF’s EC T/TA System, the Office of Planning, Research, and Evaluation (OPRE) is designing a utilization-focused evaluation to document and assess the processes, resources, and services constituting the system.
Both national and regional stakeholders constitute the EC T/TA system. Nine national centers (Centers) provide the foundation of knowledge and practice for the EC T/TA System. These Centers develop high-quality, evidence-based resources and practices, and provide training and technical assistance. As part of a coordinated national T/TA system, responsibility for disseminating the evidence-based practices, resources, and materials developed by the Centers is carried out by Head Start and Child Care T/TA specialists working in partnership with the Centers. Regional Head Start T/TA Specialists and a new Child Care State Capacity Building Center support implementation of research-based practices to address Head Start grantee and State Child Care and Development Fund (CCDF) Administrator needs, respectively. These approaches aim to build capacity and create sustainable early childhood practices at the regional, state and local levels.
The current information collection request is for gathering preliminary information from key T/TA system stakeholders within the national Centers, the 12 regions that make up the Head Start Regional T/TA network, and the 10 regions served by the Child Care State Capacity Building Center. The results will inform the evaluation design and data collection instrument development for the EC T/TA cross-system evaluation, in addition to the development of tools/resources to support Center-specific and local evaluation activities. A draft conceptual framework will also be presented to these three groups to obtain their critical feedback.
There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Approach
The purpose of the current information collection request is for approval to gather additional information about ACF’s EC T/TA system, to explore with T/TA system stakeholders their dissemination and communication preferences, to document their use of independent data collection systems for tracking T/TA delivery, and to learn what necessary supports they require in order to participate in a future utilization-focused evaluation. In addition, a draft conceptual framework of the EC T/TA System will be presented to stakeholder groups for review and to collect critical feedback for its finalization. After this information-gathering phase, ACF will submit an additional information collection request package for future data collection for the study, including data collection surveys and protocols.
There is currently limited written information available on these topics. For this reason, the study team is requesting permission to contact key stakeholders of the system to gather information about current practices and preferences. The information collection will take place in 2016 to ensure that the study design can be finalized by the end of the year, and a future data collection can begin in 2017. The remainder of this section describes the study team’s plans for contacting respondents and how the information will be used.
Research Questions
In addition to assessing the needs among key T/TA system stakeholders for research and evaluation supports, this data collection request will inform the following study research questions:
What T/TA resources and methods are currently used in the T/TA system, and how can they be improved?
How is information disseminated across the T/TA system, and how can those methods be improved?
Study Design
This collection serves as a preliminary step to gather information that will inform the design of a future EC T/TA cross-system evaluation study. The study team will utilize information from multiple sources, including an extensive literature review, National Center work plans, RFPs for regional T/TA delivery, and program experts at ACF. The study team will use information provided by ACF to identify and recruit respondents for the study from the nine National Centers, the Office of Head Start’s (OHS) Regional T/TA Coordinators, as well as the Office of Child Care’s (OCC) Child Care State Capacity Building Center Project Director and T/TA State System Specialists. Our goal is for this process to result in a census of OHS and OCC national and regional T/TA providers.
Our preference is to interview a single representative of each team, but if any of these participants request, the contractor will interview another member of their team along with or instead of the invited participant (e.g., in addition to the lead evaluator of a National Center, we would interview the Center Director or an additional member of the evaluation team). The relevant, knowledgeable respondents may vary across the nine Centers and Regional-level T/TA teams. Some, but not all, teams may have other staff who could answer the questions or who want to contribute to the interviews about goals, activities and evaluation efforts. We will not be able to identify these other individuals or know exactly how many will want additional staff involved without communicating first with the respondents identified above (i.e., Center lead evaluators, Regional T/TA Coordinators, Child Care State System Specialists). The number of respondents we have estimated in this request reflect these unknowns by including what we believe to be the maximum number of respondents likely.
These individuals also will be invited to provide feedback on the study’s conceptual framework, along with other key stakeholders from the national and regional levels of the EC T/TA system.
Upon OMB approval of the information collection instruments, the study team will email Center evaluators and T/TA coordinators and specialists a request for a 90-minute group telephone discussion. The email will introduce the study and its goals, the team that is conducting the study on ACF’s behalf, and offer suggested times for a telephone meeting (Appendix A). The telephone discussion is voluntary. The study team will lead the telephone meeting using a semi-structured interview protocol (Appendix B). The study team will answer any questions about the study and ask about both select programmatic information, such as the use of region-specific data collection systems for tracking T/TA delivery, and for input on future evaluation activities, such as preferred communication methods and needs for evaluation support. The protocol is designed to collect the minimum information necessary to allow us to understand the variation of T/TA programming across the system, and to determine whether particular study design options and activities will be feasible given variations in T/TA practices and preferences across regions and Offices (i.e., OCC and OHS).
In addition to initial telephone discussions, the study team proposes to present a draft conceptual framework of the EC T/TA system and gather feedback from key system stakeholder groups. For this purpose, the study team will host a series of up to six focus groups to present a draft conceptual framework and elicit feedback using a semi-structured discussion guide (Appendix D). Three key stakeholder groups will be invited to participate via email (Appendix C): National Center leaders and evaluators; the T/TA coordinators and staff from the 12 OHS Regions; and the Project Director and T/TA State System Specialists and staff from the Child Care State Capacity Building Center’s 10 regions.
The questions in both protocols are primarily open-ended questions, with some yes/no questions. Analysis of the qualitative data collected through the telephone interviews and focus groups will be completed by the study team to identify themes, key concepts, and areas of consensus across respondents. The limited number of questions requiring yes/no responses can be aggregated across respondents, but will primarily serve to confirm or reject interpretations by the study team and prompt further discussion. Analyses of all these data will be summarized in a Memo for ACF’s internal use, and all information collected will be used solely to inform the design and development of ACF’s EC T/TA system evaluation study, the conceptual framework, and resources and strategies to support evaluation efforts among T/TA system stakeholders.
Universe of Data Collection Efforts
There are two main data collection efforts involved with the current request for approval.
(1) Semi-Structured Interview Protocols (Appendix B1 and B2). For this collection, the study team will conduct a 90-minute phone call using a semi-structured interview protocol to interview the following groups: 1) National Center evaluators (up to three people for each of the nine National Centers); 2) OHS Regional T/TA Coordinators and staff (up to three people for each of the twelve regions); 3) OCC T/TA State System Specialists and staff (up to three people for each of the ten regions); and 4) the Project Director of the Child Care State Capacity Building Center. For this collection, the study team will also utilize a recruitment email (Appendix A1 and A2).
(2) Discussion Guide for Focus Groups on Draft Conceptual Framework (Appendix D). In addition, the study team will conduct a series of six 90-minute focus groups (in-person or via webinar) to present a draft conceptual framework and elicit feedback using a semi-structured discussion guide. The focus groups will involve three invited groups: (1) National Center leaders and evaluators; (2) OHS Regional T/TA Coordinators and staff; and OCC T/TA State System Specialists and staff; and the Project Director of the Child Care State Capacity Building Center. To maximize individuals’ participation within each discussion, we will limit the number of participants to 15 for each of the focus groups. For this collection, the study team will also utilize a recruitment email (Appendix C).
A3. Improved Information Technology to Reduce Burden
The burden on T/TA system stakeholders is minimal, and the study team plans to use information technology wherever possible. When information is available from existing documentation, such as work plans or RFPs, they will be used to supplement requests for information. To the extent possible, discussions will be completed by telephone and webinar to reduce burden on T/TA system stakeholders.
A4. Efforts to Identify Duplication
To our knowledge, systematic information about components of the current T/TA system services and operations is not available. The study team has reviewed existing documentation, examined existing T/TA-tracking data systems, and spoken with ACF program staff. All of the information needed to address the study’s research questions has not been obtainable elsewhere.
A5. Involvement of Small Organizations
The respondents in the study will include three sets of T/TA system stakeholders: (1) National Center leaders and evaluators; (2) OHS Regional T/TA Coordinators and staff; and (3) OCC T/TA State System Specialists, staff, and, additionally, the Project Director of the Child Care State Capacity Building Center. A subset of the respondents may be employees of small businesses. Burden will be minimized for respondents by restricting the interview and focus group length to the minimum required, by conducting telephone and webinar discussions at times convenient for the respondents, and by requiring no record-keeping or written responses on the part of the respondents. Also, the impact, if any, on small businesses or other small entities will be reduced by the voluntary nature of the data collection.
A6. Consequences of Less Frequent Data Collection
Each of these efforts (telephone interviews and focus groups) is a one-time data collection; participants will only be asked to volunteer for a single phone interview and attend a single focus group.
A7. Special Circumstances
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995)), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on September 15, 2014, Volume 79, Number 178, page 54985, and provided a 60-day period for public comment. During the notice and comment period, the government did not receive any comments in response to the Federal Register notice.
Consultations are being held with ACF program staff within OHS and OCC who maintain the EC T/TA system in drafting the data collection protocols and draft conceptual framework to be fielded under this request.
A9. Incentives for Respondents
No incentives for respondents are proposed for this information collection.
A10. Privacy of Respondents
As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements. Respondents are not considered human subjects, but they will still be informed of all planned uses of data, that their participation is voluntary, that some of the information they provide may be shared with ACF to help us design the study, and that any information they request to be kept private will be kept private to the extent permitted by law.
The information collected from these efforts will be used to inform the design and development of ACF’s EC T/TA system evaluation study, the conceptual framework, and resources and strategies to support evaluation efforts among T/TA system stakeholders. To inform this effort, we will share information with those who participated in the information collection, ACF Federal Staff, and a Technical Expert Panel to confirm interpretations, inform final evaluation design and evaluation TA. Responses will not be attributed to individuals. Respondents will be informed that responses will be summarized and identifiable information will not be attributed to their responses.
A11. Sensitive Questions
There are no sensitive questions in this data collection.
A12. Estimation of Information Collection Burden
Respondents will speak with a study team member about their preferences for dissemination and communication, evaluation support needs, and region-specific data collection and T/TA tracking systems, if any. These persons will not incur any expense other than the time spent answering questions.
Total Burden Requested Under this Information Collection
Instrument |
Total/Annual Number of Respondents |
Number of Responses Per Respondent |
Average Burden Hours Per Response |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
|||
Semi-Structured Interview Protocols |
94 |
1 |
1.5 |
141 |
$31.13 |
$4,389.33 |
|||
Semi-Structured Discussion Guide for Focus Groups |
90 |
1 |
1.5 |
135 |
$31.13 |
$4,202.55 |
|||
Estimated Annual Burden Total |
276 |
|
$8,591.88 |
||||||
Total Annual Cost
To compute the total estimated annual cost, the total burden hours were multiplied by the estimated average hourly wage for local program coordinators (see table above). According to the Bureau of Labor Statistics’ Current Population Survey 2015, the median hourly wage for full-time employees over age 25 with a bachelor’s degree or higher is $31.13.
A13. Cost Burden to Respondents or Record Keepers
There are no additional costs to respondents.
A14. Estimate of Cost to the Federal Government
The total cost for the data collection activities under this current request will be $56,946. Annual costs to the Federal government will be $56,946 for the proposed data collection under this OMB clearance number (0970-0356).
A15. Change in Burden
This is a new data collection.
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication
The information collected will be used primarily to plan the research design and subsequent data collection efforts for the EC T/TA Cross-System Evaluation Project. As a part of the recruitment process, initial emails will be sent to key stakeholders starting upon OMB approval to request their participation in a telephone discussion. Telephone discussions for the purpose of information gathering will take place starting in upon OMB approval and continue over the following month. Emails requesting participation in focus groups will be sent about a month following the telephone recruitment emails and focus groups with key stakeholders to gather feedback on the draft conceptual framework will occur over the following two months.
The study team will summarize the analysis of information collected through telephone interviews and focus groups, primarily by identifying major themes, key concepts, and illustrative examples, in a Memo for ACF’s internal use within thirty days of completing data collection. The information collected from these efforts will be used solely to inform the design and development of ACF’s EC T/TA system evaluation study, the conceptual framework, and resources and strategies to support evaluation efforts among T/TA system stakeholders. To inform this effort, we will share information with those who participated in the information collection, ACF Federal Staff, and a Technical Expert Panel to confirm interpretations, inform final evaluation design and evaluation TA. There are no plans to publish the information collected under this request.
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
| File Type | application/msword |
| Author | DHHS |
| Last Modified By | Molly |
| File Modified | 2016-04-29 |
| File Created | 2016-04-27 |