Anchor
Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Touchpoints for Addressing Substance Use Issues in Home Visiting: Performance Measurement Pilot
Pre-testing of Data Collection Activities
0970 - 0355
Supporting Statement
Part A
February 2021
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officer:
Nicole Denmark
Type of Request: this information collection request is for generic information collection under the umbrella generic, Pre-testing of Data Collection Activities (0970-0355).
Description of Request: The generic information collection request is for an six-month pre-testing data collection effort, including data collection, analysis, and reporting, to test the use of two new performance measures—Screening for Substance Misuse (SUD-1) and Follow-up for Caregivers At Risk of Substance Misuse (SUD-2)—that assess substance use and follow-up care among caregivers enrolled in maternal, infant, and early childhood home visiting programs (MIECHV). MIECHV awardees provide home visiting services for pregnant women and parents of young children. The services help prevent child abuse and neglect, support positive parenting, improve maternal and child health, and promote children’s development. The purpose of the pilot study is to gather information about the feasibility of collecting substance misuse screening and follow-up performance measures using SUD-1 and SUD-2 and the usefulness of those measures in improving services for families with substance misuse problems. This is a descriptive study. Two MIECHV state awardees, six local implementing agencies (LIAs), and representatives from each home visiting model implemented by the LIAs will be recruited to participate in the pilot study, which includes data collected through a Measures Reporting Tool (Instrument 1) and semi structured interviews (Instruments 2 to 4). We do not intend for this information to serve as the principal basis for public policy decisions.
A1. Necessity for Collection
In fiscal year 2016, nearly 13 percent of newly enrolled families in maternal, infant, and early childhood home visiting programs (MIECHV)-funded programs reported having household members with a history of substance misuse or who have been identified as needing substance use services (Health Resources and Services Administration, n.d.). Further, 31 percent of home visiting participants reported binge alcohol use in the three months before pregnancy or illegal drug use (including abusing prescriptions drugs) in the last month before pregnancy (Duggan et al., 2018). Parents’ substance misuse can affect children’s outcomes directly (in the case of prenatal substance use) and indirectly through impaired parenting. Home visiting programs are well positioned to reach families at risk of or experiencing substance misuse, and the programs can play an important role in engaging and supporting families to prevent, identify, and address these issues. More information is required, however, about how to integrate best practices for working with families on substance misuse issues into programs. This proposed generic information collection (GenIC) will inform the development of measures to address this gap in information.
There are no legal or administrative requirements that necessitate this collection. The Administration for Children and Families (ACF) is undertaking the collection at the discretion of the agency.
A2. Purpose
The objectives of this pre-testing data collection are to gather information on the feasibility of collecting and reporting on two potential performance measures (Screening for Substance Misuse (SUD-1) and Follow-up for Caregivers At Risk of Substance Misuse (SUD-2), the perceived usefulness of these measures for improving home visiting services, and the supports that state awardees and local implementing agencies (LIAs) require to collect and report the measures.
This proposed GenIC meets the primary goals of ACF’s generic clearance for pre-testing (0970-0355): to develop and test information collection instruments and procedures. The information collected is meant to contribute to the body of knowledge on ACF and Health Resources and Services Administration (HRSA) programs; who administer MIECHV programs. The findings from the pilot will be used to refine the way the two potential performance measures, SUD-1 and SUD-2, are defined and to produce information about implementation supports that would help MIECHV state awardees and staff at LIAs to engage and support families with substance misuse more effectively. The information might also be used in a future Home Visiting Collaborative Improvement and Innovation Network that HRSA might convene to accelerate improvements in processes and outcomes related to behavioral health, including substance misuse. The information collected is not intended to be used as the principal basis for a decision by a federal decision maker and is not expected to meet the threshold of influential or highly influential scientific information.
This information collection will explore 17 broad research questions across three criteria: feasibility, usability, and implementation supports (Table A.1). Table A.2 provides a crosswalk between the data collection instruments and research questions.
A.1. Research questions by criterion
Criterion |
Research Questions |
Feasibility |
|
|
|
|
|
|
|
|
|
|
|
Usability |
|
|
|
|
|
|
|
|
|
Implementation Supports |
|
|
|
|
|
|
|
|
|
We anticipate that pre-testing data collection activities, including data collection, analysis, and reporting, will run about six months. LIAs participating in the pre-test will receive information about the two measures and a Measures Reporting Tool, as well as training on implementing the measures in the month before the start of data collection. The LIAs will implement practices and track their progress using the SUD-1 and SUD-2 Measures Reporting Tool (see Instrument 1 for the reporting tool) over a four month data collection period1. Contractor staff will interview MIECHV state awardee administrators and LIA home visiting staff by phone to gather feedback on their experiences using information from SUD-1 and SUD-2 (see instruments 2-4 for the interview protocols). Contractor staff will ask about practices related to existing screening and follow-up care for substance use as well as screening measurements for substance use. The contractor will also interview home-visiting-model representatives by phone to understand how the SUD-1 and SUD-2 measures align with current home-visiting-model recommendations and existing requirements for substance use screening and follow-up services. See Appendix C for the specific research questions addressed through the instruments. Interviews will take place over a one-month period, currently estimated for summer 2021. LIAs will submit the SUD-1 and SUD-2 Measures Reporting Tool during the final month of data collection.
We will recruit two MIECHV state awardees and six LIAs from those two states to participate in the pilot. The contractor plans to focus on state awardees that (1) implement two or more home visiting models using MIECHV funds, (2) do not currently require all LIAs to screen for substance misuse with a validated tool, (3) have initiatives planned or underway to address substance misuse (suggesting they might be interested in participating in the pilot), and (4) anticipate having LIAs that are interested in participating in the pilot and have the capacity to do so.
We will target LIAs that have a mix of experience levels with screening caregivers for substance misuse. Specifically, we will aim to recruit two LIAs that currently universally screen caregivers for substance misuse with a validated tool, two LIAs that currently screen caregivers for substance misuse but do not screen universally or with a validated tool, and two LIAs with limited or no experience screening caregivers for substance misuse. We will prioritize LIAs that serve at least 100 families and implement more than one home visiting model. We will also aim to include LIAs that operate in both urban and rural locations.
To understand challenges that might arise with records management and data quality, we will request that LIAs report information related to calculating SUD-1 and SUD-2 on all primary caregivers who enroll in the program during the pilot period.2 To report information related to SUD-1 and SUD-2, some LIAs may need to adopt new substance use screening processes. Other LIAs may already screen for substance use issues and may not need to adopt new processes. For each enrollee, we will ask LIAs to extract from existing records (1) the date when the primary caregiver enrolled in the home visiting program; (2) whether the primary caregiver enrolled with an identified substance use disorder; (3) the dates when substance misuse screening was conducted; (4) the name of the screening tool(s) used; and (5) if applicable, the dates when the enrollee received follow-up care with a qualified behavioral health provider.
These data will be reported using the SUD-1 and SUD-2 Measures Reporting Tool—an Excel file with fields for the required data elements to ensure uniform submission of data across LIAs. We anticipate the data sources used by LIAs for abstracting the required data elements into the SUD-1 and SUD-2 Measures Reporting Tool will include electronic administrative records, paper records, and other program management records. LIAs will submit the data to the contractor via a secure network site. The contractor will conduct a webinar for participating LIAs to guide them on how to use the SUD-1 and SUD-2 Measures Reporting Tool. The contractor will also use internal staff with clinical substance use experience to support the delivery of technical assistance to LIAs. Technical assistance will include support around the technical specifications for SUD-1 and SUD-2 and use of the SUD-1 and SUD-2 Measures Reporting Tool. The information in the SUD-1 and SUD-2 Measures Reporting Tool will be used to better understand data quality and completeness issues that impact the usefulness of SUD-1 and SUD-2 and might indicate a need for technical assistance.
Table A.2 provides a detailed overview of the information collection. Appendix C presents the specific research questions addressed through Instruments.
Table A.2. Information Collection
Data Collection Activity |
Research Question Number(s) |
Instrument(s) |
Respondent, Content, Purpose of Collection |
Mode and Duration |
Performance measures data collection |
3, 17 |
Instrument 1: SUD-1 and SUD-2 Measures Reporting Tool |
Respondents: LIA managers, data managers, or both Content:
Purpose:
|
Mode: Electronic submission to a secure website Duration: 0.25 hours per reporting tool |
Semistructured interview |
2-16 |
Instrument 2: Interview Protocol: State Awardees |
Respondents: MIECHV state administrators
Content:
Purpose:
|
Mode: Phone Duration:1.5 hours |
Semistructured interview |
1-16 |
Instrument 3: Interview Protocol: LIA Managers and Data Managers, Home Visiting Supervisors, and Home Visitors |
Respondents: LIA managers, data managers, or both; LIA home visiting supervisors; LIA home visitors Content:
Purpose:
|
Mode: Phone Duration: 1.5 hours - LIA manager or data manager interview
1 hour - LIA home visiting supervisor interview and LIA home visitor group interviews |
Semistructured interview |
6, 10, 14-16 |
Instrument 4: Interview Protocol: Home-Visiting-Model Representatives |
Respondents: Home-visiting-model representatives Content:
Purpose:
|
Mode: Phone |
No other data sources or prior data collections are associated with this request.
A3. Use of Information Technology to Reduce Burden
The SUD-1 and SUD-2 Measures Reporting Tool will be a Microsoft Excel file submitted using an encrypted file-transfer protocol for sharing the data files in a highly secure manner. Only study participants and contractor staff using a password can access the files.
Information technology, such as computerized interviewing, is not well-suited to collect the intended information. Phone interviews offer the best opportunity to tailor interviews to the specific respondent with minimal burden. With respondent permission we will record interviews.
Data collected for this study cannot be found anywhere else. This will be the first study to test these substance misuse measures.
The data collection will include state and local agencies. The contractor will minimize burden for respondents by keeping the interviews as brief as possible, interviewing at times convenient for the respondent, and collecting only the necessary data required for the intended use.
This is a one-time data collection.
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the generic clearance for pre-testing activities. This notice was published on October 20, 2017; Volume 82, Number 202, page 48820, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.
The contractor consulted with Dr. Allison West, an assistant professor in the population, family, and reproductive health department at John’s Hopkins University. Dr. West is an expert on the health and well-being of expectant families and families with young children facing multiple complex adversities. Dr. West specifically provided feedback on data availability and data collection processes regarding LIAs. Dr. West was asked about her knowledge of what LIAs collect and how they store data, with a focus on LIAs in Maryland, one of her areas of expertise.
There will be no tokens of appreciation provided to survey respondents.
Because of the small number of caregivers expected to enroll for services with each LIA, date of enrollment could potentially be used to identify individuals. Date of enrollment is needed to verify whether screening occurred within 30 days of enrollment. Information will not be maintained in a paper or electronic system from which data are directly or indirectly retrieved by an individuals’ personal identifier.
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the contractor will comply with all federal and departmental regulations for private information. To inform discussion summaries, phone interviews will be recorded. Respondents will be informed of the recording process and oral permission will be obtained before the start of the interview.
As specified in the contract, the contractor will protect respondents’ privacy to the extent permitted by law and comply with all federal and departmental regulations for private information. The contractor has developed a data safety and monitoring plan that assesses all protections of respondents’ personally identifiable information. The contractor will ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor who perform work under this contract or subcontract are trained on data privacy issues and comply with the above requirements. All contractor staff involved in the project have completed training on data privacy issues. The training, completed annually, includes (1) limitations of disclosure; (2) safeguarding the physical work environment; and (3) storing, transmitting, and destroying data securely. All contractor staff sign the contractor’s Confidentiality Agreement, complete online security awareness training when they are hired, and receive annual refresher training thereafter.
As specified in the contract, the contractor will use Federal Information Processing Standard–compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. LIAs will securely transfer the Measures Data Collection tool to the contractor via a platform such as Box, which is an enterprise cloud-based solution to securely share files and collaborate. Data will be encrypted in transit and at rest. In addition, backups of data received are retained for only 60 days, which complies with data destruction requirements. The contractor will securely generate and manage encryption keys to prevent unauthorized decryption of information in accordance with the Federal Information Processing Standard. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology requirements and other applicable federal and departmental regulations. In addition, the contractor will not record any information, field notes, or any other documents that may contain sensitive or personally identifiable information on paper. All notes and documents will be stored electronically on the contractor’s secure server for the study, ensuring secure storage and limits on access. All data will be destroyed at the conclusion of the project.
The SUD-1 and SUD-2 Measures Reporting Tool collects the necessary data elements to calculate the measures, including information on presence of a substance use disorder, risk for a substance use disorder, and whether follow-up with a behavioral health specialist regarding risk for a substance use disorder occurred. While the Measures Reporting Tool will not collect names or demographic information, it will include date of enrollment, which, given the small expected sample size could be used to identify individuals. It is necessary to collect information on these sensitive topics because SUD-1 and SUD-2 could be used as performance measures of MIECHV awardees who provide programming on preventing substance misuse among caregivers enrolled in their programs. It is necessary to understand the accuracy and completeness of the data required to calculate SUD-1 and SUD-2 before the measures are implemented on a larger scale.
HML Institutional Review Board (IRB) Research & Ethics approved this pre-testing data collection effort on December 4, 2020.
Table A.5 shows the estimated burden on respondents for the data collection instruments. Estimates are based on other projects with similar instruments. The total estimated burden is 73 hours, based on the following assumptions for each data collection instrument:
Instrument 1: SUD-1 and SUD-2 Measures Reporting Tool. We estimate that there will be 10 primary caregivers per LIA and that it will take 0.25 hours to complete a reporting tool for each primary caregiver by each of the six LIA managers or data managers, for an estimated annual burden of 15 hours (6*10*0.25).
Instrument 2: Interview Protocol: State Awardees. We estimate that it will take 1.5 hours to complete each interview, for a total annual burden of 3 hours (2*1.5).
Instrument 3: Interview Protocol: LIA Managers and Data Managers, Home Visiting Supervisors, and Home Visitors.
We estimate it will take an average of 1.2 hours—(1.5+1+1)/3—to complete each interview. The home visiting model representative interviews include a universe of questions and a subset of questions will be chosen based on the specific type of respondent. We estimate the following expected completion times:
LIA Manager and Data Manager Interview – 1.5 hours
Home-Visiting-Supervisor Interview – 1 hour
Home Visitor Group Interviews – 1 hour
Six LIA managers or data managers, six home visiting supervisors, and five home visitors per six home visitor groups will participate for a total of 42 participants: 6 + 6 + (5*6).
The estimated total annual burden for all data collection activities for Instrument 3 is 50 hours (42*1.2).
Instrument 4: Interview Protocol: Home-Visiting-Model Representatives. ACF estimates it will take up to 45 minutes to complete each of the six interviews, for a total annual burden of 5 burden hours (6*0.75).
The estimated annual cost is $2,540.40.
Table A.5. Total Burden Requested Under this Information Collection
Instrument |
No. of Respondents (total over request period) |
No. of Responses per Respondent (total over request period) |
Avg. Burden per Response (in hours) |
Total Burden (in hours) |
Average Hourly Wage Ratea |
Total Annual Respondent Cost |
Instrument 1: SUD-1 and SUD-2 Measures Reporting Tool |
6 |
10 |
0.25 |
15 |
$34.80 |
$522.00 |
Instrument 2: Interview Protocol: MIECHV State Awardees |
2 |
1 |
1.5 |
3 |
$34.80 |
$104.40 |
Instrument 3: Interview Protocol: LIA Managers and Data Managers, Home Visiting Supervisors, and Home Visitors |
42 |
1 |
1.2 |
50 |
$34.80 |
$1,740.00 |
Instrument 4: Interview Protocol: Home-Visiting-Model Representatives |
6 |
1 |
.75 |
5 |
$34.80 |
$174.00 |
Total |
|
|
|
73 |
|
$2,540.40 |
a To calculate the annualized cost to respondents for the hour burden, we assume that the typical respondents will be social scientists and people in community and social services occupations. Using the mean hourly wages from the U.S. Bureau of Labor Statistics’ May 2019 National Occupation Employment and Wage Estimates (https://www.bls.gov/oes/current/oes_nat.htm#19-0000) for social scientists and related workers ($42.32) and community and social services occupations ($27.27), we calculated a mean hourly wage of $34.80 for all respondents in this data collection.
LIAs will be offered $250 to offset the costs of site participation in the pilot. Some LIAs will need to adopt a new screening process. For all LIAs, collecting data at periodic intervals, and submitting the data using the SUD-1 and SUD-2 Measures Reporting Tool, will result in data-collection or reporting costs.
The total cost for the pilot data collection activities will be $139,973 (Table A.6). This includes costs for recruiting and engaging participants, developing the pilot study plan and instruments, obtaining OMB and IRB approval, and conducting the pilot study and collecting data.
Table A.6. Estimated Total Cost by Category
Cost Category |
Estimated Costs |
Recruit and engage participants |
$40,010 |
Develop pre-testing data collection plan and instrument design |
$31,458 |
Submit OMB, IRB, and Certificate of Confidentiality |
$11,755 |
Conduct BSC and pre-testing data collection activities |
$56,750 |
Total/annual costs over the request period |
$139,973 |
This is an individual information collection under the umbrella generic clearance for pre-testing (0970-0355).
The data collection activities, including data collection, analysis, and reporting, will take place for six months following OMB’s approval. During data collection, preliminary analysis of data will occur on a rolling basis as data are received. The contractor expects to complete a pilot study report, which will deidentify participants, one month after completing analysis. ACF may share the report with participating sites and other external stakeholders.
No data sets will be shared publicly. Table A.7 presents the timeline for data collection, analysis, and reporting for the pre-testing data collection effort.
Table A.7. Data collection activities timeline
Data collection activities |
Number of months |
Data collection |
4 |
Analysis |
1
|
Reporting |
1 |
No exceptions are necessary for this information collection.
Attachments
Instrument 1: SUD-1 and SUD-2 Measures Reporting Tool
Instrument 2: Interview Protocol: State Awardee
Instrument 3: Interview Protocol: LIA Managers and Data Managers, Home Visiting Supervisors, and Home Visitors
Instrument 4: Interview Protocol: Home-Visiting-Model Representatives
Appendix A: Pilot Study FAQs
Appendix B: Invitations to Participate in the Study
Appendix C: Research Questions Addressed by Interview Protocols
Duggan, A., Portilloa, X. A., Filene, J. H., Crowne, S. S., Hill, C. J., Lee, H., & Knox, V. (2018). Implementation of evidence-based early childhood home visiting: Results from the Mother and Infant Home Visiting Program Evaluation (OPRE Report 2018-76A). Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. https://www.acf.hhs.gov/sites/default/files/opre/mihope_implementation_execsummary_2018_10_26_508.pdf
Health Resources and Services Administration. (n.d.). The Maternal, Infant, and Early Childhood Home Visiting Program: Partnering with parents to help children succeed. Health Resources and Services Administration. https://mchb.hrsa.gov/sites/default/files/mchb/MaternalChildHealthInitiatives/HomeVisiting/pdf/programbrief.pdf
National Academies of Sciences, Engineering, and Medicine. (2016). Parenting matters: Supporting parents of children ages 0–8. National Academies Press. https://doi.org/10.17226/21868
Neger, E. N., & Prinz, R. J. (2015). Interventions to address parenting and parental substance abuse: Conceptual and methodological considerations. Clinical Psychology Review, 39, 71–82. https://doi.org/10.1016/j.cpr.2015.04.004
Singer, E., Groves, R. M., & Corning, A. D. (1999). Differential incentives: beliefs about practices, perceptions of equity, and effects on survey participation. Public Opinion Quarterly, 63, 251–260.
Singer, E., & Kulka, R. A. (2002). Paying respondents for survey participation. In M. Ver Ploeg, R. A. Moffitt, & C. F. Citro (Eds.), Studies of welfare populations: Data collection and research issues. Panel on data and methods for measuring the effects of changes in soc ial welfare programs (pp. 105–128). National Academy Press. https://doi.org/10.17226/10206
Singer, E., & Ye, C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science, 645(112).
1 Data collection will last about four months. Data collection activities, including data collection, analysis, and reporting will last about six months.
2 An estimated 10 primary caregivers per LIA will enroll in the program during the pilot period.
3 Examples of sensitive topics include social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating, and demeaning behavior; critical appraisals of other people with whom respondents have close relationships, such as family, pupil–teacher, or employee–supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities that indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians, and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (such as unemployment, the Special Supplemental Nutrition Program for Women, Infants, and Children; or the Supplemental Nutrition Assistance Program); and immigration or citizenship status.
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | 1MathU Report |
Author | Anuja Pandit |
File Modified | 0000-00-00 |
File Created | 2024-07-25 |