Using Administrative Data to Answer and Inform Program and Policy Decisions

Child Care Administrator Interview about Using Administrative Data to Answer and Inform Program and Policy Decisions

Interview Protocol

Hello [Insert Name]. Thank you so much for taking the time to speak with me today. My name is [PRIMARY INTERVIEWER] and I’m joined by [SECONDARY INTERVIEWER]. I will be leading the interview today and [SECONDARY INTERVIEWER] is going to take notes for the call. We both work at Child Trends, an independent research organization. Child Trends has been contracted by the Administration of Children and Families’ Office of Research Planning and Evaluation to conduct interviews with all of the state and territory child care and development fund administrators. The purpose of this interview is to learn more about the different ways states and territories may be using their administrative data, the barriers encountered when trying to use administrative data, and suggestions for resources or supports that could be helpful. OPRE staff will use the information from these interviews to plan future activities that could support the use of early childhood administrative data.

When we refer to “administrative data” we mean information about individual children, families, and/or providers of child care and early education and other family benefits that is collected and maintained as part of the operation of government programs. The use of administrative data could range from generating data to report to ACF to analyzing program-level data to understand whether the program is meeting its goals or linking data from multiple programs or agencies to examine policy questions of interest, such as whether children from low-income families are receiving health care and high-quality early care and education services.

There are no wrong or right answers to the questions. Our main goal is to understand how state child care administrators use their data or may want to use their data. We expect the interview to take about 30-45 minutes depending on your responses. We would like to record our interview with you so that we can review your responses after the interview to be sure that we accurately captured the information you share with us. We will delete the recording once our notes are complete.

The information you share as part of these interviews will be used for planning purposes only by OPRE and Child Trends. None of the information collected during the interview will be shared publicly. Your individual responses will not be shared with anyone outside of the Child Trends study team in such a way that your identity could be revealed. Summaries of the interviews will describe findings in general terms. Your participation is completely voluntary, and you may skip any questions that you do not feel comfortable answering and can stop the interview at any time. An agency may not conduct and a person is not required to respond to an information collection request unless it displays a currently valid OMB control number. The OMB number for this collection is 0970-0356 and the expiration date is January 31, 2015.

Do you have any questions before we get started?

If it is okay, I will turn the recorder on now. Do we have your permission to record this interview? Y/ N

If yes, Turn on recorder

Questions about Respondent

First, I would like to ask you a few short questions about your current position.

1. Do you serve as the child care and development fund child care administrator for your state? YES/NO

   1. If YES, what year did you begin serving in that role? _____ year (ex. 2001)

   2. If NO, what is your current position? ______________________

      1. What year did you begin serving in that role? ________ year (ex. 2001)

If there are multiple individuals participating in the call, please note their names and positions.

Experiences Using Administrative Data

Next, we would like to hear about the different ways administrative data have been used in the past 2 years in your state or territory. The types of administrative data a state uses could include information about children participating in early care and education programs (ECE), Head Start, state pre-kindergarten or early intervention. Other sources of administrative data could include datasets related to education, child welfare, public benefits (e.g., TANF, SNAP, Medicaid), or population statistics. We are interested in hearing about the ways your administrative data have been used—within your agency and by others outside of your agency. We have developed some general questions about possible ways your administrative data may have been used, but we also want you to share other ways we may not have included.

For efficiency, we use the word “you” in the questions that we ask—but we want to be clear that we are interested in learning how your agency uses administrative data. The word “you” may also mean “you and your research partners.” We realize that sometimes state agencies are part of a research partnership in which an outside research organization (such as a university or other organization whose role is doing research) works together with state agency staff to analyze administrative data to answer questions of interest to you.

For the first set of questions, please think about examples of ways you have used administrative data in the past 2 years.

2. Have you used administrative data to create standard reports to help with agency planning? For example, a state may use administrative data to provide an automatic report that is produced each month about the types of child care used by children receiving child care subsidies. YES/NO

   1. If YES, please give an example of how you used a standard report from administrative data for agency planning.

      1. Did you encounter barriers that made this process difficult? YES/NO

         1. If YES, Please describe the barriers.

            1. What supports or resources would be helpful to address these barriers?

   2. If NO, would you like to in the future? YES/NO

      1. If YES, please describe how you would like to use administrative data for agency planning

         1. What supports or resources, if any, would you need?

      2. If NO, why not?
         
         
         

3. Have you used administrative data to respond to external data requests from other agencies or organizations? For example, a state legislative committee may ask for administrative data to show the impact of proposed budget changes or legislation. YES/NO

   1. If YES, please give an example of how you used administrative data to respond to external data requests.

      1. Did you encounter barriers that made this process difficult? YES/NO

         1. If YES, Please describe the barriers.

            1. What supports or resources would be helpful to address these barriers?

   2. If NO, would you like to in the future?

      1. If YES, please describe how you would like to use administrative data to respond to data requests your agency receives.

         1. What supports or resources, if any, would you need?

      2. If NO, why not?
         
         
         

4. Sometimes, states have questions they want to answer beyond what is required for state and federal reporting—and beyond what are included in standard reports. These questions may be about what works and why. Or they may be questions to inform policy, services or outcomes. They sometimes require more complicated research techniques and statistics. For this interview, I’m going to use the term “research” to describe these kinds of questions. In the last 2 years, has your agency’s administrative data been used to address a research question? YES/NO

   1. If YES, please provide an example.

      1. Did you encounter barriers that made this process difficult? YES/NO

         1. If YES, Please describe the barriers.

            1. What supports or resources would be helpful to address these barriers?

   2. If NO, would you like to in the future? YES/NO

      1. If YES, is there a particular research question you would like to address?

         1. What supports or resources, if any, would you need?

      2. If NO, why not?
         
         
         

5. Sometimes states combine data from different programs within their agency to address questions. For example, a state might combine subsidy and QRIS data to find out the number of subsidy children served by programs at each rating level. In the last 2 years, have you combined data from different programs within your agency to answer a particular question?

   1. If YES, please tell us about an example.

      1. What data—and from what programs—were combined within your agency?

      2. Were there barriers in combining the data? YES/NO

         1. If YES, Please describe the barriers.

            1. What supports or resources would be helpful to address these barriers?

   2. If NO, would you like to in the future? YES/NO

      1. If YES, is there a particular question you would like to address that would require combining data? What is the question?

         1. What supports or resources, if any, would you need?

      2. If NO, why not?

6. Sometimes states combine data from their agency with data from another agency. For example, a state might combine health and subsidy data to determine how many children receiving child care subsidies also participate in the state health insurance program. Or a state might connect pre-k data with K-12 data. In the last 2 years, has the administrative data from your agency been combined with data from another agency to answer a particular question? YES/NO

   1. If YES, please tell us about an example.

      1. What data—and from what agency—were combined with your own agency’s data?

      2. Were there barriers in combining the data? YES/NO

         1. If YES, Please describe the barriers.

            1. What supports or resources would be helpful to address these barriers?

   2. If NO, would you like to in the future? YES/NO

      1. If YES, is there a particular question you would like to address that would require combining data? What is the question? What data from what agencies would be needed?

         1. What supports or resources, if any, would you need?

      2. If NO, why not?
         
         
         

7. Some states might also combine data with another state. For example, a state may compare the effect of different policies focused on supporting the continuity of care for children receiving a child care subsidy. Have you combined administrative data from your state with data from another state to answer a question of common interest? YES/NO

   1. If YES, please tell us about the question, the states involved, and the data used.

      1. Did you encounter barriers that made this process difficult? YES/NO

         1. If YES, Please describe the barriers.

            1. What supports or resources would be helpful to address these barriers?

   2. If NO, would you like to in the future? YES/NO

      1. If YES, is there a particular question you would like to address? Is there a particular state you would like to work with?

         1. What supports or resources, if any, would you need?

      2. If NO, why not?

8. Have you used administrative data in any other ways that we have not already discussed? YES/NO

   1. If YES, please provide examples of the other ways you have used administrative data in your state.

      1. Did you encounter any barriers that made this process difficult? YES/NO

         1. If YES, Please describe the barriers.

            1. What supports or resources would be helpful to address these barriers?

   2. If NO, are there other ways you would like to use your administrative data that we have not already discussed? YES/NO

      1. If YES, please describe the other ways you would like you use your administrative data?

         1. What supports or resources, if any, would you need?

Thank you for your responses to the first set of questions. In the next set of questions, I would like to find out more about the resources that are available to you and your agency to help you analyze administrative data.

Research Support and Resources

The analysis of administrative data can be an attractive option for agencies that have limited funds to support additional data collection for research and evaluation. Because the data are collected as part of regular program operations, they would be collected regardless of whether any research or evaluation projects were conducted, and research costs are greatly reduced. However, to conduct analyses of administrative data some specific resources are needed.

We would like to find out more about the resources that are currently available to you to conduct analyses of administrative data.

State/territory child care administrators sometimes use different strategies for analyzing administrative data. Sometimes they have staff in their agency analyze the data, and sometimes they contract with someone else, like a research organization, to help them with more complex research design and analyses. We would like to learn more about how you determine when to work with agency staff and when to ask someone outside of your agency to analyze administrative data to answer questions of interest to you.

9. Is there a position in your agency dedicated to research and evaluation? YES/NO

If yes,

1. How many research/evaluation positions are there?

2. How accessible to you is the research staff? Do you have access to research staff in your agency when you need them?

For the next few questions about who helps you analyze data, we have organized the questions around the different uses that we asked about in the earlier questions.

I’d like you to answer the first couple of questions about using administrative data to create a standard report. A standard report is a report that is generated on a regular, ongoing basis (e.g., a monthly report of the percentage of children by age who receive child care subsidies).

10. In the past 2 years, have you asked an agency staff member to create a standard report using administrative data? YES/NO

If YES

1. How did you decide which member of your staff would oversee this task?

11. In the past 2 years, has your agency contracted with an organization or individual to create a standard report?

1. Could you please tell me why you decided to use a contractor (rather than someone in your agency)?

I’d like you to answer the next few questions about using administrative data to respond to external requests for information (like from the legislature).

12. In the past 2 years, have you asked an agency staff member to analyze administrative data to respond to an external request? YES/NO

If YES

2. How did you decide which member of your staff would oversee this task?

13. In the past 2 years, has your agency contracted with an organization or individual to analyze administrative data to respond to an external request? YES/NO

If YES

2. Why did you decide to use a contractor?

For the next few questions, please think about using administrative data to address more complex research questions. These might require advanced statistics or collecting new data to analyze with administrative data.

14. In the past 2 years, have you asked an agency staff member to analyze administrative data to address more complex research questions? YES/NO

If YES

3. How did you decide which member of your staff would oversee this task?

15. In the past 2 years, has your agency contracted with an organization or individual to analyze administrative data to address more complex research questions? YES/NO

If YES

3. Why did you decide to use a contractor?

4. What kinds of research questions were addressed? And what kinds of administrative data were utilized?

For this last set of questions about this, please think about combining data from multiple programs, agencies, or states.

16. In the past 2 years, have you asked an agency staff member to combine and analyze administrative data across programs or agencies to answer a particular question? YES/NO

If YES

4. How did you decide which member of your staff would oversee this task?

17. In the past 2 years, has your agency contracted with an organization or individual to combine and analyze administrative data to answer a particular question? YES/NO

If YES

5. Why did you decide to use a contractor?

18. The ability to conduct additional analyses of administrative data or to partner with other organizations can often be dependent on funding. Have you responded in the past 2 years to any Request for Proposals from foundations, state, or federal agencies that would provide funding to help you analyze administrative data? YES/NO

1. If YES

   1. Please describe the types of RFPs you responded to.

   2. Who wrote the proposal(s)—someone in your agency or a partner?

   3. Did you receive funding from any of your proposals?

   4. If the proposals were not funded, what do you think would help your agency be more successful in responding to RFPs for data analysis?

2. If NO

   1. Are you interested in applying for RPPs to analyze your administrative data in the future?

   2. Do you have someone in your agency or a partner you work with that has the expertise to respond to a Request for Proposals (RFP) to obtain funding to analyze administrative data?

   3. What supports or resources would help you to respond to RFPs to obtain funding to analyze your administrative data?

Other Resources and Supports

19. In addition to what we have already discussed, are there challenges that you have encountered in trying to use administrative data to answer questions?

20. Are there lessons learned or strengths in your agency related to analyzing administrative data that you would like to share with other state administrators? If so, please briefly tell us about them.

21. In addition to the resources or supports that you have already discussed, are there any other things that would help your agency answer questions with administrative data more often—or more easily?

22. Is there anything else you would like to share about using administrative data?

Thank you very much for taking the time to talk with us today about your agency’s use of administrative data. We will summarize the information and share it with our team and the Office of Planning, Research, and Evaluation to help us all think about how best we can support you in using your state data to answer questions of importance to you.

9