Attachment A: Examples of Approved Instruments under 0970-0356
Instrument: Interview Protocol
Hello
[Insert Name]. Thank you so much for taking the time to speak with me
today. My name is [PRIMARY INTERVIEWER] and I’m joined by
[SECONDARY INTERVIEWER]. I will be leading the interview today and
[SECONDARY INTERVIEWER] is going to take notes for the call. We both
work at Child Trends, an independent research organization. Child
Trends has been contracted by the Administration of Children and
Families’ Office of Research Planning and Evaluation to conduct
interviews with all of the state and territory child care and
development fund administrators. The purpose of this interview is to
learn more about the different ways states and territories may be
using their administrative data, the barriers encountered when trying
to use administrative data, and suggestions for resources or supports
that could be helpful. OPRE staff will use the information from these
interviews to plan future activities that could support the use of
early childhood administrative data.
When we refer to “administrative data” we mean information about individual children, families, and/or providers of child care and early education and other family benefits that is collected and maintained as part of the operation of government programs. The use of administrative data could range from generating data to report to ACF to analyzing program-level data to understand whether the program is meeting its goals or linking data from multiple programs or agencies to examine policy questions of interest, such as whether children from low-income families are receiving health care and high-quality early care and education services.
There are no wrong or right answers to the questions. Our main goal is to understand how state child care administrators use their data or may want to use their data. We expect the interview to take about 30-45 minutes depending on your responses. We would like to record our interview with you so that we can review your responses after the interview to be sure that we accurately captured the information you share with us. We will delete the recording once our notes are complete.
The information you share as part of these interviews will be used for planning purposes only by OPRE and Child Trends. None of the information collected during the interview will be shared publicly. Your individual responses will not be shared with anyone outside of the Child Trends study team in such a way that your identity could be revealed. Summaries of the interviews will describe findings in general terms. Your participation is completely voluntary, and you may skip any questions that you do not feel comfortable answering and can stop the interview at any time. An agency may not conduct and a person is not required to respond to an information collection request unless it displays a currently valid OMB control number. The OMB number for this collection is 0970-0356 and the expiration date is January 31, 2015.
Do you have any questions before we get started?
If it is okay, I will turn the recorder on now. Do we have your permission to record this interview? Y/ N
If yes, Turn on recorder
Questions about Respondent
First, I would like to ask you a few short questions about your current position.
Do you serve as the child care and development fund child care administrator for your state? YES/NO
If YES, what year did you begin serving in that role? _____ year (ex. 2001)
If NO, what is your current position? ______________________
What year did you begin serving in that role? ________ year (ex. 2001)
If there are multiple individuals participating in the call, please note their names and positions.
Experiences
Using Administrative Data
Next, we would like to hear about the different ways administrative data have been used in the past 2 years in your state or territory. The types of administrative data a state uses could include information about children participating in early care and education programs (ECE), Head Start, state pre-kindergarten or early intervention. Other sources of administrative data could include datasets related to education, child welfare, public benefits (e.g., TANF, SNAP, Medicaid), or population statistics. We are interested in hearing about the ways your administrative data have been used—within your agency and by others outside of your agency. We have developed some general questions about possible ways your administrative data may have been used, but we also want you to share other ways we may not have included.
For efficiency, we use the word “you” in the questions that we ask—but we want to be clear that we are interested in learning how your agency uses administrative data. The word “you” may also mean “you and your research partners.” We realize that sometimes state agencies are part of a research partnership in which an outside research organization (such as a university or other organization whose role is doing research) works together with state agency staff to analyze administrative data to answer questions of interest to you.
For the first set of questions, please think about examples of ways you have used administrative data in the past 2 years.
Have you used administrative data to create standard reports to help with agency planning? For example, a state may use administrative data to provide an automatic report that is produced each month about the types of child care used by children receiving child care subsidies. YES/NO
If YES, please give an example of how you used a standard report from administrative data for agency planning.
Did you encounter barriers that made this process difficult? YES/NO
If YES, Please describe the barriers.
What supports or resources would be helpful to address these barriers?
If NO, would you like to in the future? YES/NO
If YES, please describe how you would like to use administrative data for agency planning
What supports or resources, if any, would you need?
If NO,
why not?
Have you used administrative data to respond to external data requests from other agencies or organizations? For example, a state legislative committee may ask for administrative data to show the impact of proposed budget changes or legislation. YES/NO
If YES, please give an example of how you used administrative data to respond to external data requests.
Did you encounter barriers that made this process difficult? YES/NO
If YES, Please describe the barriers.
What supports or resources would be helpful to address these barriers?
If NO, would you like to in the future?
If YES, please describe how you would like to use administrative data to respond to data requests your agency receives.
What supports or resources, if any, would you need?
If NO,
why not?
Sometimes, states have questions they want to answer beyond what is required for state and federal reporting—and beyond what are included in standard reports. These questions may be about what works and why. Or they may be questions to inform policy, services or outcomes. They sometimes require more complicated research techniques and statistics. For this interview, I’m going to use the term “research” to describe these kinds of questions. In the last 2 years, has your agency’s administrative data been used to address a research question? YES/NO
If YES, please provide an example.
Did you encounter barriers that made this process difficult? YES/NO
If YES, Please describe the barriers.
What supports or resources would be helpful to address these barriers?
If NO, would you like to in the future? YES/NO
If YES, is there a particular research question you would like to address?
What supports or resources, if any, would you need?
If NO,
why not?
Sometimes states combine data from different programs within their agency to address questions. For example, a state might combine subsidy and QRIS data to find out the number of subsidy children served by programs at each rating level. In the last 2 years, have you combined data from different programs within your agency to answer a particular question?
If YES, please tell us about an example.
What data—and from what programs—were combined within your agency?
Were there barriers in combining the data? YES/NO
If YES, Please describe the barriers.
What supports or resources would be helpful to address these barriers?
If NO, would you like to in the future? YES/NO
If YES, is there a particular question you would like to address that would require combining data? What is the question?
What supports or resources, if any, would you need?
If NO, why not?
Sometimes states combine data from their agency with data from another agency. For example, a state might combine health and subsidy data to determine how many children receiving child care subsidies also participate in the state health insurance program. Or a state might connect pre-k data with K-12 data. In the last 2 years, has the administrative data from your agency been combined with data from another agency to answer a particular question? YES/NO
If YES, please tell us about an example.
What data—and from what agency—were combined with your own agency’s data?
Were there barriers in combining the data? YES/NO
If YES, Please describe the barriers.
What supports or resources would be helpful to address these barriers?
If NO, would you like to in the future? YES/NO
If YES, is there a particular question you would like to address that would require combining data? What is the question? What data from what agencies would be needed?
What supports or resources, if any, would you need?
If NO,
why not?
Some states might also combine data with another state. For example, a state may compare the effect of different policies focused on supporting the continuity of care for children receiving a child care subsidy. Have you combined administrative data from your state with data from another state to answer a question of common interest? YES/NO
If YES, please tell us about the question, the states involved, and the data used.
Did you encounter barriers that made this process difficult? YES/NO
If YES, Please describe the barriers.
What supports or resources would be helpful to address these barriers?
If NO, would you like to in the future? YES/NO
If YES, is there a particular question you would like to address? Is there a particular state you would like to work with?
What supports or resources, if any, would you need?
If NO, why not?
Have you used administrative data in any other ways that we have not already discussed? YES/NO
If YES, please provide examples of the other ways you have used administrative data in your state.
Did you encounter any barriers that made this process difficult? YES/NO
If YES, Please describe the barriers.
What supports or resources would be helpful to address these barriers?
If NO, are there other ways you would like to use your administrative data that we have not already discussed? YES/NO
If YES, please describe the other ways you would like you use your administrative data?
What supports or resources, if any, would you need?
Thank you for your responses to the first set of questions. In the next set of questions, I would like to find out more about the resources that are available to you and your agency to help you analyze administrative data.
Research Support and Resources
The analysis of administrative data can be an attractive option for agencies that have limited funds to support additional data collection for research and evaluation. Because the data are collected as part of regular program operations, they would be collected regardless of whether any research or evaluation projects were conducted, and research costs are greatly reduced. However, to conduct analyses of administrative data some specific resources are needed.
We would like to find out more about the resources that are currently available to you to conduct analyses of administrative data.
State/territory child care administrators sometimes use different strategies for analyzing administrative data. Sometimes they have staff in their agency analyze the data, and sometimes they contract with someone else, like a research organization, to help them with more complex research design and analyses. We would like to learn more about how you determine when to work with agency staff and when to ask someone outside of your agency to analyze administrative data to answer questions of interest to you.
Is there a position in your agency dedicated to research and evaluation? YES/NO
If yes,
How many research/evaluation positions are there?
How accessible to you is the research staff? Do you have access to research staff in your agency when you need them?
For the next few questions about who helps you analyze data, we have organized the questions around the different uses that we asked about in the earlier questions.
I’d like you to answer the first couple of questions about using administrative data to create a standard report. A standard report is a report that is generated on a regular, ongoing basis (e.g., a monthly report of the percentage of children by age who receive child care subsidies).
In the past 2 years, have you asked an agency staff member to create a standard report using administrative data? YES/NO
If YES
How did you decide which member of your staff would oversee this task?
In the past 2 years, has your agency contracted with an organization or individual to create a standard report?
Could you please tell me why you decided to use a contractor (rather than someone in your agency)?
I’d like you to answer the next few questions about using administrative data to respond to external requests for information (like from the legislature).
In the past 2 years, have you asked an agency staff member to analyze administrative data to respond to an external request? YES/NO
If YES
How did you decide which member of your staff would oversee this task?
In the past 2 years, has your agency contracted with an organization or individual to analyze administrative data to respond to an external request? YES/NO
If YES
Why did you decide to use a contractor?
For the next few questions, please think about using administrative data to address more complex research questions. These might require advanced statistics or collecting new data to analyze with administrative data.
In the past 2 years, have you asked an agency staff member to analyze administrative data to address more complex research questions? YES/NO
If YES
How did you decide which member of your staff would oversee this task?
In the past 2 years, has your agency contracted with an organization or individual to analyze administrative data to address more complex research questions? YES/NO
If YES
Why did you decide to use a contractor?
What kinds of research questions were addressed? And what kinds of administrative data were utilized?
For this last set of questions about this, please think about combining data from multiple programs, agencies, or states.
In the past 2 years, have you asked an agency staff member to combine and analyze administrative data across programs or agencies to answer a particular question? YES/NO
If YES
How did you decide which member of your staff would oversee this task?
In the past 2 years, has your agency contracted with an organization or individual to combine and analyze administrative data to answer a particular question? YES/NO
If YES
Why did you decide to use a contractor?
The ability to conduct additional analyses of administrative data or to partner with other organizations can often be dependent on funding. Have you responded in the past 2 years to any Request for Proposals from foundations, state, or federal agencies that would provide funding to help you analyze administrative data? YES/NO
If YES
Please describe the types of RFPs you responded to.
Who wrote the proposal(s)—someone in your agency or a partner?
Did you receive funding from any of your proposals?
If the proposals were not funded, what do you think would help your agency be more successful in responding to RFPs for data analysis?
If NO
Are you interested in applying for RPPs to analyze your administrative data in the future?
Do you have someone in your agency or a partner you work with that has the expertise to respond to a Request for Proposals (RFP) to obtain funding to analyze administrative data?
What supports or resources would help you to respond to RFPs to obtain funding to analyze your administrative data?
Other Resources and Supports
In addition to what we have already discussed, are there challenges that you have encountered in trying to use administrative data to answer questions?
Are there lessons learned or strengths in your agency related to analyzing administrative data that you would like to share with other state administrators? If so, please briefly tell us about them.
In addition to the resources or supports that you have already discussed, are there any other things that would help your agency answer questions with administrative data more often—or more easily?
Is there anything else you would like to share about using administrative data?
Thank you very much for taking the time to talk with us today about your agency’s use of administrative data. We will summarize the information and share it with our team and the Office of Planning, Research, and Evaluation to help us all think about how best we can support you in using your state data to answer questions of importance to you.
Instrument: Discussion Guide
INTRODUCTION
Thank you very much for agreeing to participate in this discussion. Your participation is very important to the study. I’m __________ and I work for Mathematica Policy Research/Chapin Hall at the University of Chicago, an independent social policy research center.
We are conducting a study for the Office of Planning, Evaluation, and Research at the Administration of Children and Families (ACF) at the U.S. Department of Health and Human Services. The major goal of the study is to develop a research-based conceptual framework that could be applied to existing or new ACF programs for at-risk youth. The framework focuses on improving the workforce career trajectories of youth to increase their ability to become self-sufficient and avoid public assistance (e.g., through education and work opportunities), and to promote their well-being and adult outcomes. When complete, the framework will inform ACF decisions about possible demonstrations and evaluations of innovative approaches to improving youth outcomes. The target population for this project is youth at risk of not becoming self-sufficient as they transition to adulthood.
We have drafted the framework and would like to talk with you about your experiences serving at-risk youth, to examine whether implementation of programs based on the framework is feasible and makes sense from your perspective. Our team will use your responses to refine the draft framework. Comments will not be attributed to specific individuals or programs, and no individuals will be quoted by name. Your participation in this discussion is voluntary.
This is a figure displaying the core elements of the conceptual framework in the order they are likely to occur [provide handout]. In other words, youth enter programs with a set of risk and protective factors that reflect their background and experiences; their needs and circumstances are assessed; and they are matched with interventions to increase resilience and human capital. The expected outcome is that the young person will transition to adulthood better equipped to become self-sufficient.
In many cases, youth will need to achieve physical safety and security and increase their resiliency before they can focus on building human capital. However, progress toward self- sufficiency might not always follow this trajectory. For example, some youth may be able to increase resiliency while developing human capital, or even begin to build human capital prior to increasing resiliency (as represented by the dotted line around the two types of evidence-informed intervention in the middle three columns). In some cases, the trauma-informed assessment might indicate that youth are already prepared for interventions to increase human capital and that interventions to increase resiliency are unnecessary.
Because youth are continually developing and encountering new challenges and circumstances, follow-up assessments would be planned during and after they complete each intervention. This is indicated by the arrow that loops back from the interventions to the assessment. In this way, the amelioration of prior needs could be assessed, and additional needs identified as youth progress toward adulthood.
In this framework, program duration would vary based on individual needs and the age at which youth enter the program. Some youth might require only a short-term intervention, such as one that provides them with work experiences. Alternatively, other youth might require a multi-year intervention that begins by addressing their basic needs and helping them cope with the effects of trauma before providing them with the educational and career training that will set them on a path towards self-sufficiency.
[If a group interview] I am going to moderate the discussion. It is really important for everyone to speak up so we can have a lively and informative discussion. It will also be helpful if you speak one at a time, so everyone has a chance to talk. We ask that you respect each other’s point of view. There are no right or wrong answers. You are the experts—we want to learn from you.
We ask that you not share the draft conceptual framework as it is still a work in progress. We will send you the final report, which will include the final conceptual framework.
We have many topics to cover during the discussion. At times, I may need to move the conversation along to be sure we cover everything. I would like to tape-record our discussion. I am taping our discussion so I can listen to it later when I write up my notes. No one besides our research team will listen to the tape. If you want to say anything that you don’t want taped, please let me know and I will be glad to pause the tape recorder. Do you/ Does anybody have any objections to being part of this interview or to my taping our discussion?
As a reminder, we want to reiterate that being part of this discussion is up to you, and you can choose not to answer a question if you wish. Being part of this discussion will not affect your employment.
The discussion will last about one hour, and we will not take any formal breaks. But please feel free to get up at any time if you need to.
Once again, thank you for agreeing to meet with us today. Are there any questions before we get started?
DISCUSSION QUESTIONS
What elements of the proposed conceptual framework has your program implemented, how were they implemented, and with what effect?
Needs assessment
Addressing the effects of trauma, when present
Strategies for building resilience
Preparation for adult self-sufficiency
What key lessons has your program learned with respect to identifying, engaging, and addressing the needs of youth at high risk?
In your opinion, what is the likelihood that at-risk youth would enroll and engage in a program that follows this framework?
Based on your knowledge and experience as a practitioner working with youth, what is the feasibility of implementing the proposed conceptual framework?
What supports or resources would be needed to incorporate a stronger focus on resilience building or human capital?
What changes in funding restrictions, reporting requirements, policies, or program procedures would be needed?
Would changes need to take place at the local, state, and/or federal level?
What, if any, changes would be needed in organizational structure, staffing, or training?
Would new organizational or agency partnerships need to be developed? Which organizations or agencies?
If resources were available to implement programs based on the framework, to what extent do you think they could enhance or strengthen currently available services? To what extent could they detract from existing services?
Are there youth populations for whom the framework is likely to be more or less useful (e.g., age groups, number of risks, severity of risk)?
What agencies or types of organizations are likely to be most suited to implementing programs based on this framework?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Molly |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |