Child Care Research and Evaluation Capacity Building Center Application for Receiving Individualized Support for Evaluation Capacity Building (ECE-RISE)

Chapter # Title of Chapter

Instrument 3: Interview Guide

Introduction and Consent

Thanks for your interest in ECE-RISE. ECE-RISE is an opportunity for your CCDF Lead Agency to receive tailored supports to answer the agency’s specific research question and develop skills to answer more on your own going forward. The Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) of the U.S. Department of Health and Human Services has contracted with Mathematica, an independent research organization with offices across the country to provide this support.

After reviewing your application, we believe your Agency may be a good match for ECE-RISE. During this interview, we would like to ask you some more questions to help us better understand your Agency.

This interview will take about 90 minutes. The goal of this interview is to get information that will help ACF select CCDF Lead Agencies to participate in ECE-RISE. Your participation is voluntary, and you have the right to stop at any time and withdraw your Agency from further consideration for this opportunity. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-0356, Exp: 02/29/2024. ACF and Mathematica will keep your responses private, but ACF will use your responses to understand your agencies’ strengths and challenges. Your name will not be shared with ACF, but your name might be identifiable because of your position.

During the interview, we will take notes about our discussion. To help us keep track of your answers, we will audio-record our conversation. No one besides our research team will listen to the recording. If you want to say anything that you don’t want recorded, please let me know and I will be glad to pause the recorder. We will delete all recordings at the end of the project. Is that okay?

Do you have any questions before we begin?

Potential approach to answering proposed research questions

To start, let’s discuss the questions of interest that you named in your application and ideas you have about how you might answer them. Before we jump in, please remember that if you participate in ECE-RISE, the ECE-RISE team will support your agency team as you work to answer a question that will help address a problem your agency faces or provide you with more information to improve policies, programs, or services. It’s okay if this research study ends up not addressing one of the questions from your application but for now, that’s what we are focusing on.

1. Of the two questions you included in your application, which one is most important to you and why?

2. Please tell us more about the information you already have that you think you can use to help answer this question. For example, tell us about [INSERT EXISTING INFORMATION MENTIONED IN APPLICATION].

3. Please tell us more about the new types of information that you think you would need to help answer this question that you mentioned in your application. Do you have ideas about how you might get this information?

4. Think about external factors that are outside your control—things like political or contextual factors. What are the key stumbling blocks (influencers) that might affect the progress of a research study?

5. [If not clear from the application, ask:] What role, if any, will the CCDF Administrator or other leadership in the Agency have in this opportunity?

Confirming areas of need

Now, let’s focus on a couple of potential areas of need for developing your research and evaluation capacity from your [NEEDS ASSESSMENT SURVEY/ECE-RISE SELF-ASSESSMENT] and application.

6. Based on your needs assessment and application, it sounds like you might benefit from support in two areas: [NAME TWO AREAS FROM STEPPING STONES IDENTIFIED AS AREAS OF NEED/SIGNIFICANT NEED]. Do you think that sounds about right? Why or why not?

Let’s talk about each area separately.

7. For [AREA 1], what are the main things holding you back? What supports do you think would help?

8. For [AREA 2], what are the main things holding you back? What supports do you think would help?

Partners and internal strengths

Next, let’s talk about engaging partners and building on your internal strengths.

In the application, you named some key partners within or outside your Agency that you want to involve in the project or in your Agency’s ongoing efforts to use and do research: [PARTNERS NAMED IN RESPONSE TO APPLICATION ITEM B1]. We’d like to better understand how these partners can help you conduct a research study and work on building your research capacity.

9. Please tell us more about the role of each partner, specifically in conducting a research project or helping you work on building research capacity in the areas we just discussed. Let’s start with [name the partner they listed first.]

10. How involved do you expect each partner to be with your Agency team in the activities of ECE-RISE?

11. In your application, you also wrote about [STRENGTHS NAMED IN RESPONSE TO APPLICATION ITEM C2] as the biggest asset or strength that your agency/team brings to this opportunity.

12. Please tell us more about how you see this asset or strength helping your team get the most out of the ECE-RISE opportunity.

Measuring success of participation in ECE-RISE

Now, let’s discuss what success would look like for you if your team participated in ECE-RISE.

13. What would success look like to you in 18 months?

14. How would you know you made progress in these areas?

Openness to capacity building process

Next, we’d like to step back and think about what the ECE-RISE process will look like, practically speaking, and how that fits in with your expectations and your day-to-day reality. We learn by doing—not just seeing what works but also understanding what doesn’t work so well. The ECE-RISE capacity building work is iterative—so, it requires a lot of back-and-forth communication while we work together on a specific action or solution, test it out, and then refine it to work best for your Agency. For example, we might work together on the process of linking data you’ve identified from another state agency or office with CCDF data so that you can conduct a specific analysis, but we would do it in a way that would make it easier for you to do it on your own in the future. We expect to schedule calls with teams for an hour at least twice a month and communicate by email between meetings to make progress on goals.

On your part—that is, your team and your agency—you will need to be willing to try new things, to take a new direction when something is not working, and to have patience when progress feels slow or uncertain.

15. Do you have any questions about the process?

16. How does that fit with your expectations? What do you see as the biggest challenge to operating this way? How do you think your team and your agency can adapt to working in this way?

Looking beyond ECE-RISE

Finally, let’s look beyond ECE-RISE and think about your longer-term ability to continue research and evaluation work.

17. Looking beyond ECE-RISE, what longer-term vision does your agency have about the use of research and evaluation to inform CCDF policies, related practices, or program management?

18. What external factors could affect your research and evaluation efforts after you’ve completed ECE-RISE, for better or worse? Think about things like policies, leadership, or other contextual factors.

Closing

Thank you so much for participating in this interview. We appreciate you taking the time despite all your other duties and will use your input as we develop research and evaluation supports for you and other CCDF Lead Agencies. We will be in touch in about six weeks to let you know if your Agency team has been selected to participate in ECE-RISE.

Thank you!