Supporting Statement B

Addressing Substance Use Disorders Among Families Involved with the Child Welfare System: Challenges and Promising Strategies: A Cross-Agency Collaboration

ASPE Generic Information Collection Request

OMB No. 0990-0421

Supporting Statement – Section B

Submitted: April 12, 2019

Program Official/Project Officer

Laura Radel

Senior Social Science Analyst

Division of Children and Youth Policy

Office of the Assistant Secretary for Planning and Evaluation

200 Independence Avenue, SW

Washington, DC 20201202-730-3927

Laura.radel@hhs.gov

Section B – Data Collection Procedures

1. Respondent Universe and Sampling Methods

Study Component 1: Rural Communities Substudy (RCS)

For the RCS, interviews with key informants will be conducted over the telephone, and site visit interviews with administrators, practitioners, and community partners will be conducted in person on-site. The interviews will follow an environmental scan and literature review, which will provide background information about key issues and promising practices that could be explored further in the key informant interviews and site visits.

Twelve key informant interviews will be conducted in two waves, with six interviews in each. Up to 24 individuals will participate in key informant interviews across both waves. These interviews will last about 60 minutes.

Wave 1 of key informant interviews—the first six—will engage local and national experts in child welfare and substance use, such as substantive experts, evaluators, program directors, and developers. For the first six interviews, the RCS contractor, in collaboration with ASPE, will identify three national experts in child welfare and substance use for participation. The RCS team will use a snowball sampling approach to identify potential participants for the three remaining key informant interviews in Wave 1. These interviews will focus on gathering input on key issues, promising strategies, and opportunities for improvement in rural communities.

Wave 2 of key informant interviews will gather detailed information about specific programs or strategies addressing substance use among child-welfare-involved families in rural communities. These interviews, along with the site visits (described below), will collect information about specific promising strategies. Participants include program directors or other program administrators. The RCS team will use its internal network of rural child welfare and substance use experts, findings from the environmental scan, discussions with the first six key informants, and consultation with ASPE and their federal partners to identify potential strategies and participants for the Wave 2 key informant interviews. Each Wave 2 interview will include up to two participants and will focus on strategies that (1) would not be cost-effective with an in-person visit, or (2) would not warrant a full site visit. For example, sites implementing promising strategies in suburban or urban communities, or strategies targeting a different population, might be more suitable for a telephone interview.

The RCS team will conduct additional interviews through six in-person site visits. These interviews will focus on gathering detailed information about specific strategies or program models that have been or could be implemented in rural communities. In consultation with ASPE, the RCS team will identify strategies that hold the most potential to improve services for child-welfare-involved families with parental substance use for implementation or adaptation in rural communities. Possible participants include program administrators, practitioners, and partners or staff from related organizations who are knowledgeable about a promising program and its implementation or operation. Site visit participants can represent a range of disciplines, such as child welfare, substance use treatment, law enforcement, legal and court systems, mental health, public health, and other systems that serve families or individuals struggling with opioids or other substances. As with the key informant interviews, the RCS team will use its internal network of content experts, findings from the environmental scans, discussions with key informants, and consultation with ASPE and their federal partners to identify potential participants for site visit interviews. After identifying one or more contacts from a site, the RCS team may also use a snowball sampling approach to identify additional participants as needed.

Each site visit will include 3 to 5 interviews, and each interview will include 1 to 4 participants, for a maximum of 20 participants per site. Across all six sites, a maximum total of 30 interviews will include a maximum total of 120 participants. Small-group interviews (those with more than one person) may be useful to provide multiple perspectives, such as with practitioners, including child welfare caseworkers or case managers from behavioral health agencies.

Study Component 2. Community Burden Substudy (CBS)

The CBS substudy will consist of in-person and telephone/online platform data collection with parents, caregivers, and community stakeholders across six counties as indicated in Exhibit B1. In total, the contractor will conduct key informant interviews (KIIs) with 108 respondents and 30 focus groups with approximately 8 participants each.

Exhibit B1. CBS Number of Respondents by Data Collection Activity

Activity	Number of Counties	Respondents per County	Total Respondents
Parent KIIs (parents with opioid misuse)	6	6	36
Kinship caregiver KIIs	6	4	24
Foster parent KIIs	6	4	24
Provider/stakeholder KIIs	6	4	24
Kinship caregiver focus groups	6	1 focus group/ 8 participants	6 focus groups/ 48 participants
Foster parent focus groups	6	1 focus group/ 8 participants	6 focus groups/ 48 participants
Provider/stakeholder focus groups	6	3 focus groups/ 24 participants	18 focus groups/ 144 participants

The CBS team will conduct 18 interviews and 5 focus groups in each county, as described in Exhibit B1. Interviews will last approximately 60 minutes and focus groups will last about 90 minutes. The CBS team will aim to conduct all data collection in person during a weeklong site visit to each of the six counties. However, we anticipate that scheduling conflicts will necessitate conducting some sessions over the phone or through online platforms after the site visit to complete all data collection.

Parent and Caregiver Data Collection

Sessions with parents and caregivers will focus on trajectories into opioid or other substance misuse, children’s exposure to violence and adverse childhood experiences (ACEs), interactions with the child welfare system, and help seeking behaviors.

Parents: Up to 36 parents will participate in KIIs across the six counties. To be eligible for participation, parents must meet the following criteria:

• Lived in target county for at least 1 year

• Have at least one child under the age of 18

• Have struggled with opioid or substance misuse during the past 5 years

• Are in recovery and/or currently pursuing treatment

  • NOTE: parents will be ineligible if they are active substance users who are not receiving treatment

• Priority will be given to parents who have been involved with the local child welfare system in the last 5 years, either receiving in-home services or having had their child/children removed from the home.

Caregivers: Up to 24 foster parents and 24 kinship caregivers will participate in KIIs across the six counties. In addition, up to 48 foster parents and 48 kinship caregivers will participate in focus groups across the counties. To be eligible for participation in KIIs and focus groups, kinship caregivers and foster parents must meet the following criteria:

• Lived in target county for at least 2 years (to ensure appropriate levels of knowledge about available services)

• Have caregiving experience in the target county

• In addition, must meet one of the following, with priority given to those meeting the criteria in the first bullet:

  • Currently caring for at least one child (under the age of 18) who has been removed from his/her home primarily due to parental opioid misuse

  • Has cared for at least one child (under the age of 18) who has been removed from his/her home primarily due to parental opioid misuse at some point in the last 5 years while living in the target county

Provider/Stakeholder KIIs and Focus Groups

Sessions with multi-sectoral service providers and key stakeholders will focus on the primary opioid and/or substance misuse issues each county is facing, the support services present in each county to help address these issues, and the primary prevention strategies being implemented to address substance misuse and/or children’s exposure to violence and adverse childhood experiences (ACEs).

Participant eligibility criteria for inclusion in provider/stakeholder interviews and focus groups are described below:

• Service provider from a key sector (e.g., child welfare, substance use treatment, etc.) of target county

  • Service providers may include child welfare agency administrators, supervisors, and frontline caseworkers; judges and court officials; family support staff; substance use treatment administrators and practitioners; and leaders in local community organizations.

  • Has worked in the target county for at least 2 years (to ensure sufficient knowledge about available services and existing collaborations)

• OR other key stakeholder identified by state officials

2. Procedures for the Collection of Information

ASPE and CDC coordinated closely on the design and procedures of each study component to minimize inconsistencies and redundancies in the data to be collected and to maximize alignment with the 2016 ASPE study. The teams worked together to develop discussion guides and data collection procedures. The semi-structured guides were designed to cover key topics, while allowing interview and focus group moderators to develop rapport with participants and to probe freely. Both teams will conduct internal trainings on the discussion guides and procedures prior to beginning data collection. Across both components, team members will audio record sessions, unless otherwise requested by participant(s). In addition, moderators will review the informed consent with all participants prior to the start of their sessions.

Study Component 1: Rural Communities Substudy

All RCS key informant and site visit interviewers will use semi-structured discussion guides. Discussion guide instruments have been developed for (1) Wave 1 key informants interviews, (2) Wave 2 key informant and site visit interviews with program administrators, (3) site visit interviews with practitioners, and (4) site visit interviews with partners or related organizations (Attachments A–D, respectively). There is no instrument that interviewers must follow verbatim. Interviewers will use the discussion guide to help ensure that they systematically cover each topic of interest while still preserving the freedom for the discussions to be led primarily by participants. The absence of a structured script helps develop rapport between the interviewer and the participant, which increases the completeness of the data. We estimate that key informant interviews will last 60 minutes and site visit interviews will last anywhere from 60 to 90 minutes depending on the participant’s availability and expertise, and the number of participants in an interview.

To ensure collection of the most appropriate data, all RCS team members leading and participating in key informant and site visit interviews will receive training on the aims of the RCS substudy and the discussion guides, as sites will vary in their strategies and approaches. Senior RCS staff will lead a total of 12 key informant interviews. Two-person teams will conduct a total of six site visits. Each site visit team will include an experienced senior site visitor and a junior site visitor. Senior site visitors have expertise in either child welfare or substance use, and will oversee all data collection and analysis activities for each site, ensuring that all interviews collect relevant data and reviewing the summary of findings. Junior site visitors will assist with scheduling, data collection, and analysis, and will have expertise or prior experience in substance use and child welfare. Discussion guides have been developed by the RCS substudy contractor in close coordination with ASPE.

To recruit participants, the contractor will send an email introducing the ASPE project and stating its overarching goal, which is to identify promising strategies to improve services for child-welfare-involved families in rural communities. The email will note any relevant connections, such as the person who made the recommendation (through snowball sampling or a mutual connection). If the first email is unanswered, a second email will be sent one to two weeks later, followed by a telephone call. After receiving a response from a potential participant, the contractor will propose dates and times for the interview, send an Outlook appointment with specific call information, and confirm participation through a reminder email before the scheduled date and time.

The RCS team will plan to audio-record and take notes during each telephone and site visit interview. Before beginning an interview, the interviewer will provide each participant a copy of the informed consent (see Attachment E). The interviewer will read the consent form aloud, if requested, and then verbally confirm that the participants agree to the interview and to the audio recording. No signatures for informed consent will be requested. The interviewer will notify participants that although their names and titles will not be used in summaries or briefs, one goal of the project is to disseminate information about promising strategies, which will likely necessitate naming the site’s organization and location. All participants will be informed that they can decline to answer any question they do not wish to answer, with no negative consequences. The RCS team intends to audio-record discussions, but participants can opt out of the recording and still participate. We will not record any session where a participant does not agree to recording. Instead, the interviewer will take notes during the interview/discussion.

All audio recordings from discussions will be stored on the RCS contractor’s secure network. Only the RCS contractor’s staff have access to this network, and only a subset of the RCS contractor’s staff (those working directly on the project) have access to the project folder on the network where recordings will be saved. The recording from the discussion will be deleted upon approval of their project deliverables.

Study Component 2: Community Burden Substudy

The CBS team developed semi-structured KII and focus group guides for (1) KIIs with parents and caregivers, (2) KIIs with providers/stakeholders, (3) focus groups with caregivers, and (4) focus groups with providers/stakeholders. In addition, each participant will complete a brief demographic form prior to participating in the scheduled KII or focus group. See Attachments F-L for all CBS instruments and informed consents. The CBS contractor worked in close coordination with CDC to develop all data collection instruments and procedures. We estimate that interviews will last 60 minutes and focus groups will last 90 minutes. All CBS team members participating in data collection will receive training on the aims of the study and the specific discussion guides to ensure collection of thorough, consistent, and appropriate data.

We will send two-person teams to each of the six counties for weeklong site visits to conduct in-person data collection. Each site visit team will include a lead site visitor who will oversee data collection and transcription for the county. All CBS team members will complete the following steps prior to beginning data collection:

• Moderators will ask participants to complete a brief demographic form prior to each in-person session; this form does not collect PII. For telephone and online sessions, moderators will complete the demographic form with participants over the telephone or ask participants to complete the form electronically prior to their sessions.

• For in-person data collection, moderators will provide a hardcopy informed consent for participants to keep; participants will not be required to provide a signature on the consent form. For telephone/online data collection, moderators will provide an electronic copy of the informed consent in advance.

• A moderator will review the informed consent aloud and obtain verbal consent to participate from each individual prior to starting the session.

• The moderator will request permission to audio record the discussion prior to starting. If a participant declines to be audio recorded, a second team member will take detailed notes during the in-person or telephone/online session.

The lead site visitor will be responsible for uploading the audio files from each county to the CBS contractor’s secure drive. Access to the project folder on this secure network will be limited to CBS team members. The lead site visitor will use the CBS contractor’s secure FTP site to transfer audio files to a professional firm for transcription. The firm will sign a confidentiality agreement. We will instruct the firm to scrub names and any other personally identifying information from the transcripts prior to sending back to the lead site visitor via the FTP site. Audio files will be deleted as requested by CDC.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Study Component 1: Rural Communities Substudy

The RCS contractor will draw on its experience recruiting participants for primary data collection and on its internal and external networks to identify mutual contacts to assist with recruitment as needed. When recruiting potential participants by email or telephone, the contractor will highlight the overarching goals of the project (namely, its potential to improve services for child-welfare-involved families in rural communities) and note any relevant connections, such as the person who made the recommendation (through snowball sampling or a mutual connection). If the initial outreach emails are unsuccessful, the contractor will follow up one to two weeks later with the potential participant by email or telephone to solicit their participation. The contractor will track its outreach attempts and follow up in a systematic fashion. After receiving a response from potential participants, the contractor will propose dates and times for the interview, send an Outlook appointment with specific call information, and confirm participation through a reminder email before the scheduled date and time. If recruiting challenges emerge, the contractor will consult and brainstorm additional recruiting strategies with ASPE.

Study Component 2: Community Burden Substudy

Participant Recruitment

Providers/Stakeholders: To recruit providers/stakeholders for KIIs and focus groups, the CBS team will follow the same procedures outlined above for the RCS team (see above). In addition, the CBS team will recruit individuals identified by state officials as key stakeholders who were not identified through the other means described. The CBS team will work closely with CDC to problem solve any issues that arise around recruitment.

Parents/Caregivers: We recognize that parents and caregivers affected by the opioid crisis and involved in the child welfare system are a particularly vulnerable population. As such, the CBS team will take great care to recruit these individuals in a manner that respects and protects their privacy, while also honoring their dignity. To identify and recruit parents and caregivers involved with the child welfare system, the CBS team will follow a model successfully employed in various child welfare research efforts and enlist the help of a local site liaison. The site liaison will be a staff member from the local child welfare agency. The CBS team will work closely with the site liaison to nominate individuals who meet study criteria. Additionally, we will post flyers at local child welfare agencies and service provider agencies to recruit for the study. In addition, we may use snowball sampling to help recruit kinship caregivers and foster parents, as these groups often have formal or informal support networks that may provide an avenue for recruitment. However, as an added measure to protect their privacy, we will not use snowball sampling to recruit parents with opioid misuse and child welfare involvement. Parents will be recruited to participate only in one-on-one KIIs while kinship caregivers and foster parents will be recruited to participate in one-on-one KIIs or focus groups.

Methods to Maximize Response Rates

KIIs: The CBS team will confirm parent/caregiver KIIs within 24 hours of the scheduled session and provider/stakeholder KIIs within 72 hours. If a participant is unavailable at the schedule time, we will work with the individual to reschedule the session. If a participant does not show for his/her scheduled session unexpectedly and does not respond to follow-up communications, we will aim to recruit another individual in his/her place.

Focus Groups: Focus groups with providers/stakeholders and caregivers will range in size from 4 to 10 participants, with a target size of 8 participants per session. To maximize response rates, the CBS team will employ several strategies.

• First, we recognize that some participants may have unexpected, last-minute conflicts with their scheduled focus groups due to varied demands, stress, and busy schedules. To prepare for this, we will seek to “oversample” by recruiting up to 12 individuals for each focus group and anticipating that two to four will not attend.

• The CBS team will reach out to all participants 24 hours prior to their scheduled focus groups to confirm participation. Individuals with scheduling conflicts, those who express interest after the optimal focus group size has been reached, and/or those who are resistant to participating in a focus group will be prioritized for individual interviews.

• When working with vulnerable populations, oftentimes recruitment is prolonged. If the length of time to recruit enough caregivers for a full focus group is prolonged, we will explore conducting triads (i.e., mini-focus groups with three individuals) with early-recruited caregivers to avoid losing participants to attrition and to provide greater flexibility.

4. Test of Procedures or Methods to be Undertaken

There are no plans to pretest the discussion guides for either study component. All topics and interview/focus group questions were developed in consultation with ASPE and CDC and were designed to build on the questions in the discussion guides for the 2016 ASPE study. During the interviews and focus groups, moderators will have flexibility to adapt questions as needed to cover the key topics of interest, while also allowing the participant(s) to lead the discussion. After the first one or two sessions using each discussion guide, moderators will assess how well the guide facilitated the collection of information to meet the needs of the substudy and will note any minor adjustments, such as emphasis on specific topics that would improve data collection.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Contractor staff for each substudy include experts in child welfare and substance use with extensive experience with qualitative data collection and analysis. These experts are listed in Exhibit B5 below.

Exhibit B5. Statistical Consultants and Data Collectors/Analyzers

Individual	Affiliation and Position	Role
Study Component 1: Rural Communities Substudy
Elizabeth Weigensberg	Mathematica Policy Research, Senior Researcher Telephone: (312) 585-3287 E-mail: EWeigensberg@mathemiatica-mpr.com	Project Director
Debra Strong	Mathematica Policy Research, Associate Director	Project quality assurance
Melissa Azur	Mathematica Policy Research, Associate Director	Senior advisor
Liz Clary	Mathematica Policy Research, Researcher	Task lead and senior site visitor
Nellie Garlow	Mathematica Policy Research, Health Research Analyst	Junior site visitor
Jung Kim	Mathematica Policy Research, Senior Researcher	Task lead and senior site visitor
Sarah LeBarron	Mathematica Policy Research, Human Services Senior Programmer	Junior site visitor
Christina Ribar	Mathematica Policy Research, Health Research Analyst	Junior site visitor
Study Component 2: Community Burden Substudy
Gary Chovnick	ICF, Director, Research Science Telephone: (206) 801-2814 E-mail: Gary.Chovnick@icf.com	Project Director
Gina Sgro	ICF, Lead Research Scientist	Lead Qualitative Investigator
Kristen Usher	ICF, Project Manager	Qualitative Investigator
Tesfayi Gebreselassie	ICF, Director, Statistics	Methodologist/ Statistician
Spencer Schaff	ICF, Research Scientist	Qualitative Investigator
Erin Ingoldsby	James Bell Associates, Senior Research Associate	Senior Child Welfare Advisor

LIST OF ATTACHMENTS – Section B

Note: Attachments are included as separate files as instructed.

• Attachment A: Instrument 1 – RCS Discussion Guide for Key Informant Interviews for Wave 1

• Attachment B: Instrument 2 – RCS Discussion Guide for Key Informant Interviews for Wave 2 and Site Visit Interviews with Program Administrators

• Attachment C: Instrument 3 – RCS Discussion Guide for Site Visit Interviews with Practitioners

• Attachment D: Instrument 4 – RCS Discussion Guide for Site Visit Interviews with Partners and Other Related Organizations

• Attachment E: RCS Informed Consent

• Attachment F: CBS KII–Parent/Caregiver

• Attachment G: CBS KII–Provider/Stakeholder

• Attachment H: CBS Focus Group Guide–Caregiver

• Attachment I: CBS Focus Group Guide–Provider/Stakeholder

• Attachment J: CBS Demographic Forms

• Attachment K: CBS Informed Consents