Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Contact After Adoption or Guardianship: Child Welfare Agency and Family Interactions
OMB Information Collection Request
0970 – 0567
Supporting Statement
Part B
May 2021
Submitted by:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, DC 20201
Project Officers:
Alysia Blandon
Jacquelyn Gross
Part B
B1. Objectives
Study Objectives
The primary objectives of this study are to examine intentional and unintentional ways public child welfare agencies 1 are in contact with, or receive information about, the well-being of children and youth who have exited foster care through adoption or guardianship. Little research exists about child and family well-being after adoption and guardianship. Understanding well-being includes a deliberate focus on understanding instability for these children and youth, and their parents or guardians.
This study will investigate the types of contact adoptive and guardianship families as well as community members have with public child welfare agencies. This will include investigating how and what information a child welfare agency receives about children and youth who have exited the foster care system through adoption or guardianship, if and how the agency tracks this information, and any challenges associated with gathering or tracking this information. These types of contacts could serve as an important source of information about children and families’ overall well-being after adoption and guardianship finalization. Ultimately, the study will identify practices that could be included in a toolkit designed to help child welfare agencies develop and/or strengthen post adoption and guardianship tracking and contact procedures.
Generalizability of Results
This study is intended to present an internally valid description of child welfare agencies’ post adoption and guardianship outreach and family contact activities. This information will not promote statistical generalization beyond the child welfare agency representatives who participate in the web survey and video-conference interviews.
Appropriateness of Study Design and Methods for Planned Uses
Because research is limited, we do not know much about contact between child welfare agencies and families after adoption or guardianship finalization, including how frequently families have contact with child welfare systems and why. This study will use a multimode data collection approach that will include a web survey with state adoption managers and/or staff responsible for guardianship families and conducting video-conference interviews with stakeholders from selected child welfare agencies to obtain in-depth knowledge of operations and protocols. The web survey is designed to gather information from a broad range of state child welfare agencies across the country about their contact with and tracking of, interactions with adoptive and guardianship families. Expanding on the content gathered in the web survey, six child welfare agencies will be selected for follow-up stakeholder video-conference interviews. Sites will be selected based on purposive, or convenience sampling, with a specific focus on selecting agencies that report innovative practices or procedures. The video-conference interviews will delve more in-depth to examine specific details about post adoption and guardianship contact procedures, information gathered, and tracking capacity. The findings from this study are not intended to be representative and will not be generalizable beyond the agencies who participate in the study. This study does not include an impact evaluation and should not be used to assess participant outcomes. Key limitations will be included in written products associated with the study.
As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
B2. Methods and Design
Target Population
The target population of interest for the two web surveys are state adoption programs managers in the U.S., a total of 50 potential respondents. Once the web surveys are initiated, six agencies will be selected for stakeholder video-conference interviews. For each of the six case study sites, the study team will conduct stakeholder interviews with up to 7 staff per agency (for a total of up to 42 interviews). These interviews will gather more in-depth information to expand upon the web survey results.
Recruitment and Site Selection
The study team will identify individual respondents to participate in 1-2 web surveys and video-conference interviews. First, the sampling process for the web survey will be described followed by the video-conference interviews.
Agency Web Survey. The study team will identify candidate web survey respondents via an existing list of state adoption program managers that is held by the Children’s Bureau and cross-check that list with the one available on the Child Welfare Information Gateway public web site. The study team will identify guardianship states through a list of approved guardianship programs held by the Children’s Bureau. In states that have both an adoption and guardianship program, the adoption program managers will receive both the adoption web survey and the guardianship web survey. Each survey will have a unique identifier that will allow the study team to track responses to a specific agency.
Stakeholder Video-conference Interview. Once the web survey is initiated, the study team will select child welfare agencies to participate in stakeholder video-conference interviews. Child welfare agencies will be selected for interviews based on the responses received from the web surveys that indicated the agency engages in one or more of the following: (1) specific post adoption or guardianship contact procedures, (2) post adoption or guardianship administrative data tracking and linkage capabilities, (3) a unique outreach or service program that specifically targets adoptive or guardianship families, or (4) site-specific information that suggests the child welfare agency has some unique contribution that other agencies could use (e.g., a creative post permanency service or practice where additional information would be informative). Staff and stakeholders from these selected child welfare agencies will be candidate for the video-conference interviews.
Stakeholder Video-conference Interviews. Up to seven 90-minute stakeholder video-conference interviews will be conducted at six selected agencies. These interviews will begin with the adoption program manager who completed the survey. The state adoption managers will then nominate other appropriate staff and stakeholders for the study team to interview. Staff and stakeholders selected for video-conference interviews could include data managers, services coordinators, and/or service providers. Identified stakeholders could also include non-child welfare agency staff (i.e., contracted service providers) who contact families that have adopted children from foster care or entered into a legal guardianship relationship.
B3. Design of Data Collection Instruments
Development of Data Collection Instruments
There are four instruments to be used in this study: the Agency Web Survey on Adoption (Instrument 1a), Agency Web Survey on Guardianship (Instrument 1b) and Stakeholder Interview Discussion Guide on Adoption (Instrument 2a) and Stakeholder Interview Discussion Guide on Guardianship (Instrument 2b).
Agency Web Survey for Adoption (Instrument 1a) and Guardianship (Instrument 1b)
The Agency Web Surveys on Adoption and Guardianship were designed to acquire across-child welfare agency general information about post adoption and guardianship contacts (initiated by agencies and families), types of information received, and capacity to track or record these contacts. The web surveys questions are highly structured. The total length of the Agency Web Survey on Adoption will be, on average, 20 minutes. The total length of the Agency Web Survey on Guardianship2 will be, on average, 15 minutes. The Agency Web Survey on Guardianship will only be completed by adoption program managers in states with a federally subsidized guardianship program (i.e., Kin-GAP).
All questions in these surveys were developed for this project. Questions were developed in consultation with experts in the field. The surveys begin by asking respondents a broad set of questions about the types of family contact their child welfare agency has. If respondents indicate they use a certain type of child welfare agency contact, the web surveys direct them to answer follow-up questions about that specific type of contact. The web surveys will cover the following topic areas:
Routine child welfare agency contact with families post adoption and guardianship (e.g., annual recertification process or outreach regarding available services or supports)
Family-initiated contact (e.g., request for services or subsidy adjustment) or community member-initiated contact
Administrative data tracking and linkage (e.g., the ability to report the frequency with which the child welfare agency is in contact with families after adoption or guardianship, and the ability to link families who are in contact with the child welfare agency back to their child welfare record)
Stakeholder Interview Discussion Guide on Adoption (Instrument 2a) and Guardianship (Instrument 2b)
Stakeholder video-conference interviews will include 90-minute stakeholder interviews, which will follow the Stakeholder Interview Discussion Guide on Adoption (Instrument 2a) or Guardianship (Instrument 2b). Questions in this discussion guide were also developed in consultation with experts in the field. All questions were developed for use in this study. The discussion guide is designed to gather more in-depth details about agency post adoption and guardianship contact procedures, information gathered, and tracking capacity. The discussion guide explores topics using open-ended questions. Interviewers will use web survey responses and information from the document review process to tailor the discussion guide questions to the individual child welfare agency staff or stakeholder. The interview discussion guide covers the following topics:
Child welfare agency-initiated routine contact with or outreach to families post adoption and guardianship (e.g. form of contact, success of contact, and information received from families who respond).
Family and community-initiated child welfare agency contact (e.g., agency response, relation to child)
Administrative data linkage (if indicated in child welfare agency web survey)
Interest in tracking post adoption and guardianship well-being outcomes
B4. Collection of Data and Quality Control
The contractor, RTI International, will be responsible for the web survey data collection part of this study. RTI’s partners at Case Western Reserve University and East Carolina University will lead the case study video-conference interviews along with an RTI project team member.
Agency Web Survey
The Agency Web Surveys’ (Instruments 1a and 1b) process will begin with an email to respondents that explains the study goals and parameters (see Attachment A: Children’s Bureau Outreach Email- Agency Web Survey and Attachment B: PAGI Team Staff Follow-Up Email -Agency Web Survey). This preliminary email will include a message from the Children’s Bureau acknowledging the importance of the survey and urging adoption managers to respond. A second email message to respondents will come from a PAGI team staff member and will include a link to the web survey, the consent process (see Attachment C: Staff Consent Form for Web Survey) and a phone number to call for any questions or concerns.3 Adoption managers who work within states with a guardianship program will receive two survey links—one for the adoption web survey and one for the survey focused on guardianship practices. We will send follow-up reminder emails once a week for 8 weeks. After two weeks, to encourage survey completion, we will also conduct telephone follow-up with survey non-responders. Respondents will have 8 weeks total to complete the web survey. The study team will use RTI’s Survey Management System to manage and control all aspects of the web survey data collection, including instrument authoring, tracing, and case management. To ensure data quality, our Survey Management System uses a standardized process to implement the web survey and track response outcomes through a unified list of events and status codes. We will use the Voxco platform for the web survey, which will help to create a single instrument, no matter what device respondents use to complete the survey. Voxco automatically detects the device (e.g., laptop, phone, tablet) a respondent is using and adapts the survey display to the device. This improves the survey experience for the respondent and can lead to higher response rates.
Agency web survey respondents will be able to start the survey and complete it later. Once they start, the system will send reminders to respondents who have not completed it. Throughout the web survey data collection period, an RTI staff member will be available to answer emails and phone calls and respond in a timely manner. Once data collection for the web survey concludes, web survey access will cease, but the call-in number and email will remain open for several months to address any additional questions or concerns.
Video-conference Interviews
After web survey data collection is initiated, the study team will conduct video-conference interviews with up to 42 stakeholders from 6 child welfare agencies. The video-conference interview process will include the following:
Document review. Prior to the video-conference interviews, the study team will ask child welfare agency staff to provide supporting documentation associated with their post adoption or guardianship contact procedures. This could include procedure manuals, recertification letters/forms, staff/provider training guides, or example child welfare agency communications (e.g., recent newsletters, outreach letters, public websites describing services). The PAGI study team will accept these documents in any format (e.g., word documents, pdfs, links to websites with procedures). Study team members will review all documents provided and record information relevant to key study questions . Information gathered from the document review will be used to streamline information gathered within the stakeholder video-conference interview.
Video-conference interviews. The study team will conduct up to 7 stakeholder interviews for each of the 6 agencies selected. Staff identified for stakeholder interviews will be nominated by the adoption program manager or an agency administrator. Stakeholder interview respondents will receive an email from a project team member to set up a video-conference interview time (see Attachment D: Site Visit Stakeholder Interview Email Outreach) and will complete a consent form (see Attachment E: Site Visit Stakeholder Interview - Consent Form) prior to their interview. Upon receiving consent and prior to conducting the video-conference interviews, the selected child welfare agency staff will be asked to provide any existing supporting documentation associated with their post adoption and/or post guardianship contact procedures.
The study team will identify a point of contact at each agency, preferably the adoption program manager, selected for video-conference interviews. The identified agency point of contact will collaborate to coordinate and schedule video-conference interviews. Once we identify a point of contact for each selected agency, the study team will host video-conference meetings with this individual to discuss the overall purpose of the video-conference interviews, identify key child welfare agency staff and stakeholders to participate in video-conference interviews, consider candidate websites or tracking databases for review, and coordinate the submission of documents for study team review.
A Study Information Sheet (see Attachment F) will be provided to all web and stakeholder interview respondents.
B5. Response Rates and Potential Nonresponse Bias
Response Rates
For the web survey of the state adoption managers, we anticipate a 70% response rate. This is based on our experience with other projects that surveyed state adoption managers (see, for example, Fuller, Bruhn, Cohen, Lis, Rolock, & Sheridan, 2006 where 49 out of 50 states responded), and because we will be enlisting the help of the Children’s Bureau. The interviews are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion; response rates will not be calculated or reported.
Nonresponse
As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.
B6. Production of Estimates and Projections
Estimates produced by this work will be prepared for internal use by ACF and external release by the agency in the form of presentations, reports, and/or publications. These publications will include information that is not generalizable to a nationally representative sample of child welfare agencies. Consequently, the data will not be used to generate overarching national population estimates, either for internal use or dissemination. Limitations to use and applicability of findings will be noted in materials made public.
B7. Data Handling and Analysis
Data Handling
The Agency Web Surveys will be programmed for web administration. The web survey technology affords several improvements in the collection of survey data specific to mitigating and correcting detectable errors and minimizing the errors typically produced by data entry by hand. First, this technology improves the consistency of data that a respondent provides. This reduces the need for subsequent data editing. Second, the web survey technology provides greater expediency for data processing and analysis. Several backend processing steps, including editing, coding, and data entry, become a part of the data collection process and the survey responses load directly into the final data file.
The stakeholder video-conference interviews will use Zoom to conduct and record interviews. Zoom technology allows for recordings and automatic transcriptions. Zoom-produced transcriptions will be reviewed carefully by a study team member to ensure accuracy. Zoom Cloud recordings and transcripts are stored for 180 days and then purged. The data from the interviews will be summarized for each site, and a case study report will be written for each site. If appropriate, the research team will use qualitative software (e.g., NVIVO) to code for themes across respondents and the six agencies
Data Analysis
The Contact After Adoption or Guardianship study design is poised to answer four research questions (RQs):
RQ 1: What contact do child welfare agencies initiate with families after adoption or guardianship, and how does this contact provide information on the well-being of the child or youth?
RQ 2: What contact do families (children, youth, parents, or guardians) initiate with child welfare agencies after adoption or guardianship?
RQ 3: How do child welfare agencies use the information gathered about families after adoption or guardianship?
RQ 4: To what extent do child welfare agencies track information about families post adoption and guardianship? What challenges do child welfare agencies experience in tracking instability formally and systematically?
The Agency Web Surveys (Instruments 1a and 1b) will provide data designed for quantitative analyses in response to these RQs, while the Stakeholder Interview Discussion Guides (Instruments 2a and 2b) will provide data designed for descriptive qualitative analysis and to generate case study video-conference interview reports with more in-depth information about agency practices.
The PAGI study team will analyze web survey data to reflect upon the proportion of web survey respondents who endorse various types of contact (child welfare agency and family-initiated) as well as various procedures for using or tracking post adoption or guardianship information. We will analyze responses separately to determine adoption versus guardianship-relevant procedures. We will produce descriptive statistics (including percentages and standard errors) to describe the reported frequencies of procedures and processes that web survey respondents endorse. We will also provide percentages of respondents who endorse various challenges to post adoption/guardianship contact. For example, in response to RQ 1, we will describe the proportion (and associated standard error) of respondents who indicated that their child welfare agency conducts annual check-ins, formal letter requests to families, and/or formal service request follow-up outreach. For agencies that endorse a specific contact procedure, we will then provide the average frequency of this contact, the percentage of this contact that is tracked systematically (RQ 4), and the percentage of various ways that agencies report using the information that results from this type of contact (RQ 3). Where possible, we will use bivariate tests (chi-square and t-tests) to examine differences in reported procedures for adoption versus guardianship families. These tests will allow us to determine, for example, whether formal annual check-ins are more likely to be reported for post adoption contact than post guardianship contact.
Stakeholder video-conference interviews will be used to develop agency case studies that describe in-depth examples of agency procedures and practices. Interviews with different agencies are intended to help describe the “why” behind certain web survey results. For instance, the web survey may find that very few respondents endorse the use of post adoption or guardianship tracking procedures. Information from stakeholder video-conference interviews may describe the challenges in implementing such tracking procedures or how the procedures may have been tried in the past (and why they failed). In other words, stakeholder interview data will supplement the results of the web survey with detailed examples of various practices. A transcript will be created for all stakeholder interview calls. Transcripts will then be coded for qualitative themes by two raters. Specifically, the raters will carefully read the transcripts to identify themes or patterns that relate to the research questions. They will create codes by identifying a word, or group of words, that describe the responses to each research question. These themes will be used to produce the synthesis report.
Data Use
ACF and RTI are committed to transparency in research. Sharing access to survey datasets will support further analyses that can inform policy and practice related to post adoption instability. We will submit the final, deidentified Agency Web Survey data to the National Data Archive on Child Abuse and Neglect (NDACAN), following their requirements for ensuring privacy. The contractor will prepare supporting materials to contextualize and assist in interpretation of the data and as required by this project dataset, including a variable list and codebook.
Web survey and video-conference interview findings may be used to develop a toolkit that child welfare systems can use. This toolkit could provide information to child welfare jurisdictions about how agency family contact, data linkages, and tracking occur in other sites.
Web survey and case study results will be summarized into two reports:
Individual case study reports. We will produce a summary for each of the 6 selected agencies of information learned from stakeholder interviews conducted specific to each agency.
Synthesis Report. We will also produce a synthesis report summarizing findings across video-conference interviews and integrating these with information gathered from the web survey. We will use information gathered from video-conference interviews to supplement the quantitative results from the web survey and include a synthesis of the lessons learned and themes across all types of information gathered.
B8. Contact Person(s)
Alysia Blandon
Senior Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
U. S. Department of Health and Human Services
330 C Street SW, 4th Floor
Washington, DC 20201
alysia.blandon@acf.hhs.gov | (202) 205-3598
Heather Ringeisen
Director, Center for Behavioral Health and Development
Survey Research Division
RTI International
Post Office Box 12194
Research Triangle Park, NC 27709
hringeisen@rti.org | 919.541.6931
Attachments
Instrument 1a: Agency Web Survey – Contact after Adoption
Instrument 1b: Agency Web Survey - Contact after Guardianship
Instrument 2a: Stakeholder Interview Discussion Guide - Adoption
Instrument 2b: Stakeholder Interview Discussion Guide - Guardianship
Attachment A: Children’s Bureau Outreach Email - Agency Web Survey
Attachment B: PAGI Team Staff Follow-Up Email - Agency Web Survey
Attachment C: Staff Consent Form Web Survey
Attachment D: Stakeholder Interview Email Outreach
Attachment E: Stakeholder Interview - Consent Form
Attachment F: Study Information Sheet
Attachment G: Public Comment
References
Fuller, T. L., Bruhn, C., Cohen, L., Lis, M., Rolock, N., & Sheridan, K. (2006). Supporting adoptions and guardianships in Illinois: An analysis of subsidies, services, and spending. Illinois: University of Illinois at Urbana-Champaign.
Stambaugh, L., Khoury, D., Domanico, R., & Ringeisen, H. (2018). Understanding post adoption and guardianship instability for children and youth who exit foster care: Compendium of policies and practices. Unpublished government report.
1 For the purpose of this study, we are only referring to public agencies. The primary scope does not include private agencies. The only time where private agencies may be relevant is within the video-conference interviews. Public child welfare agencies may describe contractual relationships with private providers and recommend some private provider representatives for stakeholder interviews.
2 Since terminology may differ by location, the beginning of the web survey will include text that clarifies the use of the term “guardianship,” which includes terms such as “custodianship.”
3 The phone number will direct respondent to responsible project team members.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Coleman, Amanda (ACF) |
| File Modified | 0000-00-00 |
| File Created | 2021-05-11 |