Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Expanding Evidence on Replicable Recovery and Reunification Interventions for Families (R3)
Formative Data Collections for ACF Research
0970 - 0356
Supporting Statement
Part A
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Calonie Gray
Kelly Jedd McKenzie
Part A
Executive Summary
Type of Request: This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for ACF Research (0970-0356).
Description of Request:
The Expanding Evidence on Replicable Recovery and Reunification Interventions for Families (R3) project will conduct phone calls and site visits with programs that use recovery coaching in a child welfare setting. The purposes of the data collection are to 1) select candidate sites for a feasibility study, and 2) conduct a feasibility study to assess sites’ readiness for a future rigorous evaluation. We do not intend for the data to be generalized to a broader population. We do not intend for this information to be used as the principal basis for public policy decisions.
A1. Necessity for Collection
Parental substance misuse has become one of the most common reasons families are involved in the child welfare system, driven in part by the opioid epidemic as well as ongoing misuse of other drugs and alcohol (Radel, Baldwin, Crouse, Ghertner & Waters, 2018; U.S. Department of Health and Human Services (HHS), 2019). Families with children in out-of-home care due to parental substance misuse have among the lowest likelihoods of reunifying. Though successful completion of substance use disorder (SUD) treatment is but one of many factors considered in court decisions to reunify children with their parents, studies show that parents who do complete treatment are more likely to reunify (Green, Rockhill, & Furrer, 2007; Smith, 2003; Choi, Huang, & Ryan, 2012). Recovery coaching is a promising approach to support parents who are working toward treatment completion, recovery, and ultimately reunification with their children when possible.
The Expanding Evidence on Replicable Recovery and Reunification Interventions for Families (R3) project is being conducted in accordance with the 2018 Substance Use-Disorder Prevention that Promotes Opioid Recovery and Treatment (SUPPORT) for Patients and Communities Act (Public Law 115-271; see Attachment A). The SUPPORT Act authorized $15 million for the replication and evaluation of an intervention utilizing coaches for families engaged in the child welfare system due to parental SUDs.
In response, in 2019, the Office of Planning, Research, and Evaluation (OPRE) partnered with the Children’s Bureau, both within the Administration for Children and Families (ACF), U.S. Department of Health and Human Services (HHS). Because ACF had not previously undertaken work in this area, together they launched the R3 project to lay the foundation for a rigorous evaluation of the effectiveness of recovery coaches to improve family reunification and SUD recovery outcomes to fulfill the legislative mandate. This proposed generic information collection will inform future evaluation efforts, for which full information collections requests will be submitted for review and approval.
A2. Purpose
Purpose and Use
inform the development of ACF research
maintain a research agenda that is rigorous and relevant
ensure that research products are as current as possible
inform the provision of technical assistance.
The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.
The purpose of this information collection is to conduct a feasibility study of one or more promising family recovery and reunification interventions that use recovery coaches. As part of the R3 project, the feasibility study will support ACF by establishing the foundation for a rigorous impact evaluation in accordance with the SUPPORT Act. It will be used by ACF to determine the viability of conducting a high-quality evaluation of the selected intervention(s) in specific sites.
Research Questions or Tests
The information collection will enable the R3 project to address the following research questions:
What is the organizational capacity of the site to implement the intervention for the period of the feasibility study and subsequent impact study?
What is the strength of program implementation and service delivery outputs?
What are the community and regional contextual factors that could affect implementation of the intervention and the impact study? What are “services as usual?”
What are the technical assistance needs of the site that would be necessary to launch an impact study?
What is the accessibility and quality of the site’s existing administrative data, and is it adequate to measure key outcomes of interest for treatment and control groups?
What is the capacity of the site to collect additional data for the impact study for treatment and control groups?
Overview of Design
In the first stage, which preceded this request, the R3 project team compiled evidence on recovery and reunification interventions that use recovery coaches in child welfare settings. Specific activities included:
Identifying recovery coach interventions that may be successful candidates for assisting families affected by SUD whose children are in or at risk of entering the child welfare system. Sources for identifying interventions included published literature, gray literature, federal grant program profiles, and recommendations from stakeholders and experts in the field.
Conducting initial screens on identified interventions based on publicly available and existing ACF-specific documents. Using a rubric developed for the study and available documentation, the R3 team conducted preliminary assessments of readiness for a rigorous evaluation that could establish the interventions as “supported” or “well-supported” under the evidence guidelines of the Title IV-E Prevention Services Clearinghouse established by ACF.
Identifying a subset of interventions--based on results of the preliminary assessment--for a more comprehensive assessment of readiness for replication and a rigorous evaluation.
In the second stage, which is the focus of this request, the project team will select one or more interventions for the feasibility study and identify sites that are currently implementing the intervention(s) or that could potentially implement them in the future. The feasibility study will assess the factors necessary to determine the viability of conducting a rigorous evaluation of the selected intervention(s) in specific sites. Specific activities include:
Conducting outreach and engagement activities to identify candidate sites currently implementing the selected intervention(s) or that may have the capacity to implement the intervention(s). Up to four sites will ultimately be selected for the feasibility study.1 A full information collection request will be submitted for a future impact evaluation.
Working closely with up to four sites in the feasibility study to determine whether participation in a rigorous evaluation is feasible, and lay the groundwork for potential future participation. Participation in an impact evaluation is not covered under this information request; rather, any site suitable for an impact evaluation will be invited after OMB approval for a future full information collection request has been received.
Data Collection Related to this Request
The project team is proposing two different engagements with potential study sites:
For the purpose of selecting potential sites for the feasibility study, the R3 team will engage program leadership at candidate sites to introduce them to the R3 project's aims and gather initial information on their fit for the feasibility study. This activity is expected to take place over 7 months following OMB approval and will culminate in the selection of up to four sites to include in the feasibility study. With each site, the team will conduct a series of phone calls and one virtual site visit. (A virtual site visit via phone calls and/or video conferences will replace the in-person site visit, given the situation with COVID-19.)
For the purpose of feasibility study data collection, the R3 team will conduct an additional site visit to the four selected sites in the second half of 2021. (We will assess the possibility of conducting in-person site visits as this timeframe approaches.)
Exhibit 1 provides more detail on the planned data collection.
Exhibit 1
Data Collection Activity |
Instrument(s) |
Respondent, Content, Purpose of Collection |
Mode and Duration |
Informational Calls with Candidate Sites |
Instrument 1: R3 Discussion Guide for Informal Informational Calls with Potential Program Sites |
Respondents: Leadership (up to two individuals) at up to 10 candidate programs or sites
Content: Study goals and benefits to participating; site role and responsibilities; resources available to reduce potential site burden. If already implementing the intervention, the program’s design and operation. If not already implementing the intervention, potential capacity to implement a new program.
Purpose: Introduce the study. Collect preliminary information needed to determine whether the site is appropriate for the feasibility study. |
Mode: Phone
Duration: 1 hour per call; up to 3 calls per site |
Virtual Site Selection Site Visits |
Instrument 2: R3 Discussion Guide for Virtual Site Selection Visits |
Respondents: Leadership and staff (up to six individuals) at up to 4 candidate sites
Content: Study goals and benefits; site role and responsibilities; local context that could affect participation; resources available to reduce potential site burden; program scale, design, operations, and maturity; technical assistance needs.
Purpose: Build on initial relationships with sites to support their participation in the feasibility study (if selected). Confirm prior information and gather additional information about program operations. Address any site concerns. |
Mode: Phone/video conference
Duration: 8 hours (split into multiple sessions) |
Feasibility Data Collection Site Visits |
Instrument 3: R3 Discussion Guide for Feasibility Data Collection Visits |
Respondents: Leadership and staff (up to 14 individuals) at up to 4 sites selected for the feasibility study
Content: Review target population; program capacity and sample sizes; settings; enrollment and retention; service environment; administrative data sources and accessibility; staffing and training; challenges and future plans; integration of study design with intake procedures; feedback on research questions and potential study designs.
Purpose: Continue to address the criteria needed to implement a rigorous evaluation design. |
Mode: In person or phone/video conference
Duration: 4-8 hours per respondent (split into multiple sessions) |
Other Data Sources and Uses of Information
The R3 team will use existing documentation it collected through a systematic scan of recovery coaching interventions for families involved in the child welfare system to supplement information collected through this request.
A3. Use of Information Technology to Reduce Burden
Due to the nature of the information being collected (notes from phone calls and site visits), there is no use of information technology to reduce burden.
A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency
The proposed data collection builds on secondary data (available information from other existing sources) that the R3 team compiled in the first phase of the project. Those sources of secondary data included information available from published literature, gray literature, federal grant program profiles, and recommendations from stakeholders and experts in the field. Direct information collection via phone calls and site visits, as proposed in this request, is necessary to fully assess potential sites’ fit for the feasibility study and conduct the feasibility study with selected sites.
A5. Impact on Small Businesses
We do not know if any candidate sites or their partner organizations are small businesses. If they are, we will ensure that site visit interviews occur at a time and place least burdensome and disruptive to their business functioning.
A6. Consequences of Less Frequent Collection
Less frequent collection would be detrimental to the quality and timeliness of the feasibility study and ACF’s ability to conduct the future, rigorous evaluation as required by the SUPPORT Act.
A7. Now subsumed under 2(b) above and 10 (below)
A8. Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.
The R3 expert consultant group provides feedback on overall study design considerations. The group is comprised of experts in child welfare and substance use fields, including non-federal experts and federal experts representing three different agencies—HHS/Substance Abuse and Mental Health Services Administration, HHS/National Institutes of Health/National Institute on Drug Abuse, and Department Of Justice/Office of Juvenile Justice and Delinquency Prevention.
A9. Tokens of Appreciation
No tokens of appreciation will be provided for this information collection.
A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing
Personally Identifiable Information
The information collection does not request any personally identifiable information.
Assurances of Privacy
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information.
Data Security and Monitoring
Abt will ensure that all of its staff, including staff of all subcontractors, who perform work under this contract are trained on data privacy issues and comply with the above requirements. The data collected through this information request will not be shared outside of the federal and contractor staff directly involved with the R3 project. All Abt staff are required to participate in annual data security awareness training.
Abt complies with the Privacy Act of 1974, Health Insurance Portability and Accountability Act of 1996 (HIPAA), and the E-Government Act of 2002, including Title III: Federal Information Security Management Act (FISMA), which covers site security, security control documentation, access control, change management, incident response, and risk management. To restrict access to project data, Abt has implemented specific access controls. Abt restricts data access to only authorized personnel with access permissions appropriate to their specific role. Abt restricts all access to data stored locally by folder, by using both role and group permissions through technologies such as Microsoft Active Directory services. For remote access, Abt requires that personnel use an Abt laptop to connect to the Virtual Private Network (VPN). Any changes to access permissions and account management are centrally managed. Abt uses NIST Special Publications (SP) 800-53 rev 4 to define and establish information security controls. Abt provides perimeter protection of project data through multiple firewalls that are configured, and Evaluation Assurance Level -certified (EAL), to restrict both inbound and outbound access. Other Abt perimeter protections include anti-spam, anti-malware, and anti-intrusion tools. Abt provides two methods for secure external access to our information systems: 1) a VPN, which is compliant with the Federal Information Processing Standard (FIPS) 140-2; and 2) a secure file transfer portal that uses Secure Sockets Layer (SSL) technology, as well as Advanced Encryption Standard (AES) encryption that is also FIPS 140-2 compliant.
A11. Sensitive Information 2
This information collection does not request any sensitive information.
A12. Burden
Explanation of Burden Estimates
For informational phone calls with candidate sites (Instrument 1), our burden estimates assume:
The team will conduct calls with up to ten (10) candidate sites;
There will be multiple calls with each site, with an estimated three (3) calls per program;
Each call will last one (1) hour;
Two (2) individuals at the program director level participate in each call.
For virtual site selection visits (Instrument 2), our burden estimates assume:
The team will conduct virtual site visits with up to four (4) candidate sites;
Each virtual site visit will be a total of eight (8) hours over phone or video conference (split into multiple shorter sessions), which includes time for sites to gather information needed to respond to questions;
A mix of individuals at the program director and staff level will participate. For each site, we assume four (4) individuals at the program director level and two (2) program staff.
For feasibility study site visits (Instrument 3), our burden estimates assume:
The team will conduct site visits (virtual or in-person) with up to four (4) sites participating in the feasibility study;
Each site visit will be a total of 16 hours, either in-person or over phone/video conference (split into multiple shorter sessions), which includes time for sites to gather information needed to respond to questions;
A mix of individuals at the program director and staff level will participate in discussions. For each site, we assume six (6) individuals at the program director level will participate for eight (8) hours each; and eight (8) program staff will participate for four (4) hours each.
Estimated Annualized Cost to Respondents
We estimate the average hourly wage for directors and leadership at candidate programs, $35.05, to be the average hourly wage of “social and community service managers” (11-9151) as determined by the U.S. Bureau of Labor Statistics National Occupational Employment and Wage Estimates. We estimate the average hourly wage for staff at the programs, $24.23, to be the average hourly wage of “counselors, social workers, and other community and social service specialists” (21-1000) as determined by the U.S. Bureau of Labor Statistics National Occupational and Wage Estimates (U.S. Department of Labor, May 2019; https://www.bls.gov/oes/current/oes_nat.htm#21-0000).
Exhibit 2
Instrument |
No. of Respondents (total over request period) |
No. of Responses per Respondent (total over request period) |
Avg. Burden per Response (in hours) |
Total/ Annual Burden (in hours) |
Average Hourly Wage Rate |
Total/ Annual Respondent Cost |
Instrument 1: R3 Discussion Guide for Informal Informational Calls with Potential Program Sites |
20 |
3 |
1 |
60 |
$35.05 |
$2,103 |
Instrument 2: R3 Discussion Topics for Virtual Site Selection Visits |
16 (program directors) |
1 |
8 |
128 |
$35.05 |
$4,486.40 |
8 (program staff) |
1 |
8 |
64 |
$24.23 |
$1,550.72 |
|
Instrument 3: R3 Discussion Topics for Feasibility Data Collection Visits |
24 (program directors) |
1 |
8 |
192 |
$35.05 |
$6,729.60 |
32 (program staff) |
1 |
4 |
128 |
$24.23 |
$3,101.44 |
|
Total |
100 |
|
|
572 |
|
$17,971.16 |
A13. Costs
There are no additional costs to respondents.
A14. Estimated Annualized Costs to the Federal Government
Cost Category |
Estimated Costs |
Instrument Development and OMB Clearance |
$67,412 |
Field Work |
$255,979 |
Analysis, Reporting, and Dissemination |
$303,245 |
Total costs over the request period |
$626,636 |
A15. Reasons for changes in burden
This is a new information collection request under the umbrella generic clearance for Formative Data Collections for ACF Research (0970-0356).
A16. Timeline
Activity or Deliverable |
Timing* |
Plan for Identifying Candidate Interventions |
November 2019 (complete) |
Report on Candidate Interventions |
July 2020 (draft complete) |
Informational Calls with Candidate Sites |
November – May 2020 |
Conduct Virtual Site Selection Visits |
November – May 2020 |
Plan for Feasibility Study |
May 2020 |
Site Recommendation Memo for Feasibility Study |
June 2021 |
Sites Finalized for Feasibility Study and MOUs Signed |
August 2021 |
Conduct Site Visits for Feasibility Study Data Collection |
August - October 2021 |
Draft Interim Report |
May 2021 |
Draft Final Technical Report |
December 2021 |
Contract ends |
March 2022 |
A17. Exceptions
No exceptions are necessary for this information collection.
Attachments
Instruments
Instrument 1: R3 Discussion Guide for Informal Informational Calls with Potential Program Sites
Instrument 2: R3 Discussion Guide for Virtual Site Selection Visits
Instrument 3: R3 Discussion Guide for Feasibility Data Collection Visits
Appendices:
Appendix A: Text from SUPPORT Act Section C. 8082
1 We define a program as a consistent implementation of an intervention with shared practices, policies, leadership, and (usually) funding. We will engage leadership at the program level in order to identify and connect with sites. We define a site as one location affiliated with and subordinate to a program. A site is the hub for program services and also the evaluation's frontline (e.g., where study enrollment and data collection occur). In large-scale studies, a single program often has many affiliated sites.
2 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |