Supporting Statement A - CCDBG Formative Generic

Supporting and Learning from Child Care and Development Block Grant (CCDBG) Implementation Research and Evaluation: Understanding the Two-Phase Grant Structure to Inform Future Research

OMB Information Collection Request

0970-0356

Supporting Statement

Part A

March 2017

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Meryl Barofsky, Senior Social Science Research Analyst

A1. Necessity for the Data Collection

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval to collect information from Child Care Lead Agencies and evaluation partners awarded CCDBG Implementation Research and Evaluation Planning Grants. This data collection is part of data collection to inform the development of OPRE research under a project titled: Supporting and Learning from Child Care and Development Block Grant (CCDBG) Implementation Research and Evaluation.

Study Background

OPRE awarded CCDBG Implementation Research and Evaluation Planning Grants to eight Child Care Lead Agencies in seven states and one territory in September 2016 and plans to announce a second cohort of grantees in April 2017. The planning grants provide up to $75,000 over 18 months and supports agencies as they form research partnerships, build their research capacity, and design research and evaluation studies that examine policies and practices related to the implementation of the CCDBG. Grantees will submit proposals for implementation grant funding at the end of their planning grant. Successful applications will be funded under a second grant phase.

OPRE aims to understand the benefits and challenges of the two-phase grant structure. OPRE proposes to collect data throughout the planning grant period to collect information on grantees’ perspectives towards the grant process, their motivations for applying, and the value of the planning grant phase. OPRE is also interested in understanding why some grantees waited to apply until the second round of funding and if the two-stage grant structure encouraged or discouraged applicants overall. OPRE assumes that having the 18-month planning period (rather than requiring a fully developed evaluation plan for the initial proposal) may have encouraged some Lead Agencies to apply that would not have otherwise. Even for lead agencies with a strong research capacity, the planning period allows additional time to identify research partners and external evaluators, explore data sources and test data management systems to determine strengths and weaknesses, conduct exploratory analyses to inform research questions and study design, and determine best methodological and statistical approaches for a full evaluation.

However, since the receipt of a planning grant does not guarantee an implementation grant, the two-stage grant structure could have some drawbacks. For example, not having stable funding may jeopardize hiring and retaining research staff across phases. Lead agencies that require a bidding process to secure an external evaluator either have to do two successive bids, or wait and bring the external evaluator on board during the second phase.

The information collected will be used for internal planning purposes to guide ACF’s future research activities. To date, ACF has never examined this issue of grant structuring for CCDF Lead Agencies in a systematic way and has limited prior knowledge to make an informed decision on the most suitable approach.

OPRE has contracted the Urban Institute to lead the Center for Supporting Research on CCDBG Implementation (henceforth referred to as “Center”) to support grantees’ and other lead agencies’ efforts to build their research capacity. Under this contract, Urban Institute will lead the data collection efforts.

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency to guide ACF’s future research activities.

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

The purpose of this data collection is to study the benefits and drawbacks of the two-stage grant structure used in the CCDBG Implementation Research and Evaluation Grants to inform ACF’s future research activities. Data collection methods include telephone interviews with grantees and evaluation partners at strategic points during the planning grant phase as well as semi-structured group discussions with grantees at the end of the planning grant.

Research Questions

The data collection aims to address the six research questions that appear in Exhibit 1. At the core of the data collection is the examination of the grant structure. The first and second research questions are central to this examination, while research questions 3, 4, and 5 contribute to our understanding of the planning process and why it is needed based on grantees’ perceptions and experiences during the planning phase. The final research question builds on the former questions to identify how grantees’ planning experiences can inform other Child Care Lead Agencies.

Exhibit 1. Research Questions and Primary Data Sources

	Data Collection Methods
Research Questions	Initial Phone Interviews	Follow-Up Phone Interview	Semi-structured Group Discussions
GRANT STRUCTURE: What are the benefits and challenges of having a two-phase grant structure with a planning period rather than awarding full research and evaluation grants? What improvements could be made to the grant structure to better support grantees? What alternative approach might work better?	X	X	X
MOTIVATIONS: What motivated grantees to apply for a planning grant? Did the two phase structure of the grant encourage grantees to apply? Why did the second cohort of grantees not apply in the first round (if they didn’t)? Why did some lead agencies choose not to apply at all?	X
PLANNING PROCESS: How are grantees utilizing planning grant resources and structuring their planning activities to design research and evaluation plans? How do grantees’ perceive their data and research capacity? What steps do grantees take to enhance their capacity for research and evaluation? How did the planning grant period support grantees?	X	X
VALUE OF PLANNING: How, if at all, do grantees evolve or strengthen their proposed research ideas during the planning phase? To what do they attribute their progress?		X	X
LESSONS LEARNED: How can grantees’ shared learning experiences inform future capacity-building efforts and research and evaluation on CCDF implementation in other lead agencies?		X	X

Study Design

There will be three data collection tasks: an initial telephone interview early in the planning grant phase, a follow-up telephone interview about one year after grant award, and an in-person semi-structured group discussion with grantees when they are gathered at a grantee meeting in Washington, DC. This grantee meeting is a grant requirement. Respondents will include state/territory/tribal child care administrators, state/territory/tribal agency staff, and evaluation partners inside or outside government.

The data collection is designed to elicit information early in the planning grant phase to capture grantees’ perceptions of their research and data capacity and motivation for applying, as well as their need for time and resources to plan their research and evaluation activities. After grantees have had time to plan, collaborate, and determine the direction of their projects, we will engage them in a second telephone interview to gather their reflections on the planning process and their perspectives towards the two-phase grant structure. The group discussion will allow for an exchange of ideas and perspectives in the context of a cohort of grantees. Group discussions like this are a useful method for identifying consensus and conflicting views that might not otherwise emerge in interviews with individual grantees.

We expect some grantees will need more time to plan than others and each will have unique strengths and challenges that facilitate or complicate their planning. We want to uncover the value of having a separate planning period and requiring grantees to compete for an implementation grant. We will collect information on their perspectives on the grant structure and whether a different grant structure or different grant requirements (e.g., shorter/longer planning grant period) would have been a better fit for their research efforts.

Universe of Data Collection Efforts

The universe of data collection efforts includes two phone interview protocols that will be administered to all planning grant recipients who consent to participate (see Attachment B and D). We expect up to 16 grantees from different states and territories. A third protocol consists of an in-person group discussion with grantees that attend a grantee meeting as part of their grant requirements and choose to participate in a semi-structured group discussion (Attachment E). The questions for the interview protocol are newly developed.

A3. Improved Information Technology to Reduce Burden

Whenever possible, advanced technology will be used to collect and process data to reduce respondent burden and make data processing and reporting more timely and efficient. We will recruit and communicate with all participants through email or by phone (see Attachments A and C for recruitment scripts), and we will conduct all interviews via telephone (see Attachments B and D for initial and follow-up telephone interview guides). We will also email a reminder with the date and time of the phone interview unless participants indicate a preference to receive this information in another format.

Attendance at a grantee meeting in Washington, DC is a grant requirement. Grantees may choose who to send to the meeting to represent their team. No additional travel is required for the proposed data collection. Since grantees will already be convened, we will maximize the opportunity to have an in-person group discussion (see Attachment E for moderator’s guide for group discussion).

During the phone interviews and group discussion, a research assistant will use a laptop to type verbatim notes. As a back-up, he or she will also use a digital audio recorder to capture responses so that respondents will be able to speak without having the interviewer interrupt or slow them down to take notes. (Participants’ verbal consent to be audio recorded will be obtained before turning on the audio recorder.)

A4. Efforts to Identify Duplication

This is the first time ACF has awarded two-stage planning and implementation grants to CCDF Lead Agencies. This data collection is unique to this grant opportunity to understand grantees’ motivations, perceptions, and experiences related to this planning grant—how this grant supports their efforts to design a rigorous evaluation to study the impacts of policy changes under new federal child care regulations.

A5. Involvement of Small Organizations

Respondents will be primarily CCDF Lead Agency staff, but evaluation partners working with state grantees may be from small organizations. Evaluation partners may choose to participate in phone interviews and the in-person group discussion (if present at the grantee meeting when data collection occurs). We will minimize burden on all respondents by scheduling the phone interviews at times convenient for them and spacing the two interviews apart by approximately eight months. Beyond the scheduled interviews and group discussion, we will not require them to prepare or submit any materials or data not already required under their grant. This will help to ensure that grantee participation does not conflict with their other responsibilities. Also, the impact, if any, on small businesses or other small entities will be reduced by the voluntary nature of the data collection.

A6. Consequences of Less Frequent Data Collection

Less frequent data collection would reduce the ability to evaluate the two-stage grant process. Collecting information from grantees at multiple points during their planning period and as they transition to the implementation phase will lead to more complete data to address the proposed research questions.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the generic clearance for information collection. This notice was published on September 15, 2014 in Volume 79, No. 178, page 54985, and provided a 60-day period for public comment. The second notice was published on January 9, 2015, Volume 80, No. 6, page 1420, and provided a 30-day period for public comment. ACF did not receive any comments.

Consultation with Experts Outside of the Study

The proposed data collection benefits from being part of a larger ACF project—the establishment of the Center for Supporting Research on CCDBG Implementation. This Center engages experts to support CCDG Lead Agencies in their efforts to build research capacity and design and carry out child care research and evaluation studies. The Center offers learning opportunities for lead agencies awarded planning grants as well as other CCDF Lead Agencies. Experts from the Center provided input into the research questions and reviewed draft interview protocols and believe the questions will elicit the desired information about the two-stage grant structure. Experts include:

• Julia Isaacs, Urban Institute

• Monica Rohacek, Urban Institute

Additionally, an evaluation expert outside the project team—Mary Bogle, Urban Institute—reviewed the data collection plan and research protocols. The team lead revised the data collection instruments to reflect input from these experts.

A9. Incentives for Respondents

No incentives for respondents are proposed for this information collection.

A10. Privacy of Respondents

Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their participation in the study and the responses they provide will not affect their competitive application for Phase 2 (implementation grant) funding (see Attachment B, D, and E for consent language provided in interview and discussion guides.)

The Contractor will not disclose to the government or any party outside the research team who participated in the data collection, or intentionally disclose any information provided by individual respondents. The Contractor will summarize the information collected from both primary data sources (e.g., interviews) and secondary data sources (e.g., grantee applications; progress reports) and submit a summary report to the government for internal use only, not for public release or broader dissemination.

Although individuals will not be cited as sources, and the research team will make every effort to safeguard data and protect respondents’ privacy, it is possible that OPRE will be able to identify the source of a quote or information provided given their familiarity with the grantees’ projects and experiences. ACF will know the grantee teams and the individuals who will likely participate in this data collection. To the extent possible, the Contractor will anonymize quotes and aggregate responses in the final internal report to reduce the possibility of disclosure. The report will focus on addressing the stated research questions and not go beyond the scope to report extraneous comments made.

The Contractor will submit the final report after OPRE makes funding decisions. The report will not be published or widely disseminated or shared with grantees. It will be an internal document for OPRE staff.

As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements.

As specified in the contractor’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard.  The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.  Information will not be maintained in a paper or electronic system from which they are actually or directly retrieved by an individuals’ personal identifier.

The Contractor’s Institutional Review Board has reviewed and approved all proposed data collection activities to ensure protections of human subjects and compliance with data security procedures. (See Attachment G for IRB Certificate of Approval.)

A11. Sensitive Questions

There are no sensitive questions in this data collection.

A12. Estimation of Information Collection Burden

Newly Requested Information Collections

The total hours for this data collection activity is estimated to be 80 hours, including time for recruitment, scheduling, and participation in the interviews and group discussion.

Total Burden Requested Under this Information Collection

The proposed information collection does not impose a financial burden on respondents. Respondents will not incur any expenses other than the time spent in conversation with the contractor. We will invite one individual from each grantee team (N=16) to participate in the initial interview and the group discussion, and two individuals from each grantee team (N=32) to participate in the follow-up interview.

Instrument	Total Number of Respondents	Annual Number of Responses	Number of Responses Per Respondent	Average Burden Hours Per Response	Annual (Total) Burden Hours	Average Hourly Wage	Total Annual Cost
Initial interview (Attachment A)	16	1	1	1.16	19	$32.61	$619.59
Follow-up interview (Attachment D)	32	1	1	1.16	37	$32.61	$1,206.57
Group discussion guide (Attachment E)	16	1	1	1.5	24	$32.61	$782.64
Estimated Annual Burden Total				80		$2,608.80

Total Annual Cost

For state administrators, an average hourly salary of approximately $32.61 is assumed

based on the Bureau of Labor Statistics (BLS) estimates for median hourly wages for management and professional state and local government workers.¹ There will be no direct cost to the respondents other than their time to participate in the study.

A13. Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A14. Estimate of Cost to the Federal Government

The total cost for the data collection activities under this current request will be $75,666.

A15. Change in Burden

This is an additional individual information collection under OMB#0970-0356.

A16. Plan and Time Schedule for Information Collection, Tabulation and Publication

Once we receive notification of OMB approval, recruitment for the initial interview will begin. The recruitment process could take anywhere from a day to a week and a half to complete per grantee. As a part of the recruitment process, an initial email will be sent to grantee team lead(s) to ask about their interest in participating in a telephone interview for the data collection. If they do not respond to that email, a second email will be sent a week later. If they still do not respond to the second email in a few days, a follow-up call will occur. If the grantee still does not respond after the follow-up call, an email will be sent to alternative team members whose contact information is on record with the grant officer (Attachment B).

The initial interviews will be scheduled over the course of several weeks, at times convenient for participants (Attachment A). Follow-up interviews will occur approximately 12-14 months after the grant award date (which was September 2016 for Cohort 1 and estimated to be April 2017 for Cohort 2; Attachment C, D).

The group discussion will occur in March 2018—the tentative date for the grantee meeting aligning with the Child Care and Early Education Policy Research Consortium annual conference (Attachment E, F).

After each data collection point, data will be cleaned, coded, and analyzed using NVivo software. Two members of the research team will independently review all qualitative responses. There will not be a public report of the findings, and the findings will not be discussed publicly. The draft report will be submitted in December 2018; the final report will be submitted by March 2019.

A17. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 U.S. Department of Labor, Bureau of Labor Statistics “National Compensation Survey: Table 1: Summary Mean
hourly earnings and weekly hours for selected workers and establishment characteristics.” 2010.

http://www.bls.gov/ncs/ocs/sp/nctb1344.pdf

10