Supporting Statement B - CCDBG Formative Generic

Supporting and Learning from Child Care and Development Block Grant (CCDBG) Implementation Research and Evaluation: Understanding the Two-Phase Grant Structure to Inform Future Research

OMB Information Collection Request

0970-0356

Supporting Statement

Part B

March 2017

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer: Meryl Barofsky, Senior Social Science Research Analyst

B1. Respondent Universe and Sampling Methods

This data collection will provide information to the Office of Planning, Research and Evaluation (OPRE) on the benefits and challenges of implementing a two-stage grant structure in which Child Care Lead Agencies must compete for both a research planning grant and a separate implementation grant. Respondents will include state/territory/tribal child care administrators, state/territory/tribal agency staff, and research partners inside or outside government who received CCDBG Implementation Research and Evaluation Planning Grants in 2016 (Cohort 1) or 2017 (Cohort 2). We expect up to 16 grantees total from different states, territories, and tribes.

All grantee teams will be invited to participate in telephone interviews and an in-person group discussion with other grantees for the purpose of this data collection. Specifically, we will invite the project team lead from the lead agency to participate in an initial phone interview (N=16). We will identify the names of these individuals in the grantees’ application materials. For the follow-up telephone interview, we will invite the same project team lead plus one additional core team member inside the agency or with an external partner organization (N=32) who is identified throughout the course of the project. If the contacted individual is not available or declines participation, he or she may nominate one other person from the project team. 

B2. Procedures for Collection of Information

As a first step, we will email the point person(s) for each team, usually a project director or project manager, and introduce them to the purpose of the study and invite them to participate in a voluntary 1-hour phone interview (see Attachment A for email script and Attachment B for interview guide including language to obtain verbal consent). We estimate that it will take approximately 10 minutes for grantees to review the email inviting them to participate as well as schedule their interviews. We will share a set of topics or guiding questions in advance of the call so grantees have a chance to reflect and prepare their thoughts. We will interview the first cohort of grantees (who consent to participation) following OMB clearance. We will contact the second cohort a month or two after their grants are awarded (tentatively June 2017).

After the first grant year has passed for each cohort, we will schedule a second 1-hour follow-up phone interview with each grantee team to gather their reflections on the value of a planning grant and their expectations for the implementation phase (see Attachment C for email script and Attachment D for interview guide). We estimate that it will take approximately 10 minutes for grantees to review the email inviting them to participate as well as schedule their interviews. These calls will be structured as small group interviews and include lead agency program staff managing the project and external evaluation partners.

We anticipate a grantee meeting will be held in DC in March 2018, following the annual Child Care and Early Education Policy Research Consortium (CCEEPRC) meeting. This marks the end of the planning grant period for the first cohort of eight grantees and the mid-point for the second cohort. We expect a total of up 16 grantees at the grantee meeting with both cohorts. We will conduct semi-structured group discussions with grantees at a scheduled time during the meeting (see Attachment E). In advance of the meeting, the Center for Supporting Research on CCDBG Implementation (which is organizing the grantee meeting) will share a meeting agenda with grantees that includes the time of the voluntary group discussions. After the draft agenda is shared, the contractor team lead will email the point person for the grantee teams (e.g., a project director or project manager) to invite him or her to participate in a group discussion or to forward the invitation on to a colleague attending the meeting who might be interested in participating (see Attachment F for invitation). The invitation will provide a description of the purpose of the group discussion, requirements for participation, and the risks and benefits. The invitation will note the voluntary nature of the data collection and that participating or refusing to participate will have no bearing on their eligibility for an implementation grant or any future grant award decisions.

We plan to divide the grantees into two groups—one for each cohort—with one representative from each grantee team. The two group discussions will occur concurrently during a 75-minute slot in the meeting agenda. Again, we estimate 15 minutes for grantees to review the invitation about this discussion. The group break-out sessions will be an opportunity for grantees to discuss their experiences together and share their perspectives on the two-stage grant structure and the value of having a planning phase. The questions we use to guide the discussion will be similar across groups; separating cohorts will help organize the discussion, since grantees will likely be at a similar planning stage as others within their cohort.

All interviews and discussions (phone and in-person) will be conducted by researchers experienced in qualitative research. At the start of the interview, the interviewer will administer all consent procedures to ensure fully-informed and voluntary participation. The interviewer will ask each participant to provide verbal consent before beginning the phone interview or in-person group discussion. The interviewer will inform participants that the interviews/discussions will be audio recorded for the purpose of filling in notes and confirming quotes. The recordings will not be shared with anyone outside the contractor team, will be saved in a secure drive at the Urban Institute, and will be deleted after the transcripts are finalized. Because the recordings will be vital to data accuracy, participants will need to provide consent to the recording to participate in the data collection.

Interviewers will follow semi-structured interview guides to capture information consistently across respondents. A senior researcher will lead the interviews while a research assistant takes typed, close-to-verbatim notes on either a laptop or desktop computer equipped with PGP encryption for data security.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

We expect at least one person from each grantee team will participate in the telephone interviews. Because the grantees are engaged in a Community of Practice and the Contractor has regular contact with them, we expect most, if not all, of those invited to participate in the data collection will consent to do so.

It is possible one or more grantee teams will not attend the in-person grantee meeting tentatively scheduled for March 2018. In that case, that team will not be represented in the semi-structured group discussion.

Dealing with Nonresponse

Although we expect at least one person from each grantee team to participate in each of the three data collection activities, it is possible one team will decline participation, not be available to complete one of the phone interviews, or not be able to attend the in-person group discussion. Because of the qualitative nature of data collection, we will look for themes across all grantees to address the proposed research questions. In our analysis, we will consider missing data and the characteristics of the grantee whose data we are missing and consider any nonresponse bias when summarizing the information.

Maximizing Response Rates

To maximize response rates, we will schedule interviews at times most convenient for respondents and be flexible in scheduling outside of normal business hours to accommodate grantees located in other time zones. We will conduct interviews by telephone to eliminate any travel burden on respondents. We expect most grantees will send at least one team member to attend the grantee meeting in Washington, DC since it is a grant requirement. We will maximize the opportunity to collect data in-person in a group discussion as grantees are already gathered.

B4. Tests of Procedures or Methods to be Undertaken

To test the interview protocols, the first interview scheduled will act as a pilot. After the first initial telephone interview, any minor yet necessary changes that need to be made to the interview protocol will be made before conducting the telephone interviews with the other grantees. Similarly, for the follow-up interview, the first grantee scheduled will serve as a pilot.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor team includes Dr. Heather Sandstrom, who will lead the interviews and manage data collection, analysis, and report writing; Amelia Coffey, research associate, who will assist with leading semi-structured group discussions, data analysis, and report writing; and Tyler Woods, research assistant, who will recruit respondents, schedule interviews, take notes, and code and analyze data.

4