SSA Measuring Success rev

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

GenIC: Measuring, Supporting, and Understanding Child and Caregiver well-being through Employment and Self-Sufficiency Research

Under

Umbrella Generic Clearance:

Formative Data Collections for ACF Research

0970 – 0356

Supporting Statement

Part A

March 2023

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Erin Cannon, Emily Ross, and Elizabeth Karberg

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a generic information collection (GenIC) under the umbrella generic, Formative Data Collections for ACF Research (0970-0356).

• Description of Request: The Measuring, Supporting, and Understanding Child and Caregiver well-being through Employment and Self-Sufficiency research (Measuring SUCCESS) project is developing a conceptual framework to describe what, why, and how to measure family social well-being in the context of the Administration for Children and Families Office of Planning, Research, and Evaluation broader Portfolio on Welfare and Family Self-Sufficiency Research. The data collection effort will include one-time focus groups with welfare and family self-sufficiency program clients who are parents and caregivers, to provide the perspective of individuals and families with lived experience. Findings from this data collection effort are not intended to be generalized to a broader population, but rather will provide one narrow perspective that will supplement other information gathering activities (a literature and conceptual model scan and synthesis, and input from government, academic, and other experts with lived experience) that all contribute to the conceptual model. Findings from the focus groups will not be generalizable to any broader population but will simply add context to the conceptual model. The conceptual model is a central deliverable for the Measuring SUCCESS project; the purpose is to develop a comprehensive conceptual framework that describes the kinds of child and family well-being outcomes, other than economic well-being, that we can expect to be improved by a family’s participation in TANF or related programs. We do not intend for this information to be used as the principal basis for public policy decisions.

A1. Necessity for Collection

The Measuring, Supporting, and Understanding Child and Caregiver well-being through Employment and Self-Sufficiency research (Measuring SUCCESS) project is conducting foundational activities to integrate a child and family social well-being¹ lens into the U.S. Department of Health and Human Services, Administration for Children and Families (ACF) Office of Planning, Research, and Evaluation’s (OPRE) Welfare and Family Self-Sufficiency Research Portfolio. This portfolio primarily studies interventions that aim to promote employment, self-sufficiency, and economic well-being among families with low incomes. However, ACF’s mission extends beyond economic outcomes to “promote the economic and social well-being of families, children, youth, individuals and communities who are resilient, safe, healthy, and economically secure.” This data collection is needed to help ACF understand child and family well-being from parents and caregivers with lived experience, which will supplement other information gathering activities (a literature and conceptual model scan and synthesis, input from government, academic, and other experts with lived experience) that all contribute to the Measuring SUCCESS project conceptual model.

The proposed parent and caregiver focus groups are one of the activities to contribute to the Measuring SUCCESS project. Other activities include: (1) cataloging and assessing the state of existing child-, parent-, and family-related measurement in OPRE’s Welfare and Family Self-Sufficiency Research Portfolio, (2) reviewing existing literature on whether and how self-sufficiency programs or employment characteristics influence child, parent, and family social well-being, (3) reviewing models of how self-sufficiency programs intend to influence child, parent, and family well-being, (4) developing a conceptual framework for understanding the social well-being of children, parents, and families in the context of welfare and family self-sufficiency research, and (5) developing a compendium of relevant measures of child, parent, and family social well-being in the context of family welfare and self-sufficiency research.

There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at the discretion of the agency. ACF has contracted with Mathematica to conduct this GenIC.

A2. Purpose

Purpose and Use

The information collected through the focus groups with parents and caregivers with lived experience proposed in this GenIC will be used to inform ACF about parents' and caregivers’ social well-being to supplement other project information gathering activities (. Although these data are not intended to be representative of the larger population of participants, it is critical to understand the perspectives of the people who are the intended recipients of employment and training services. The data collection efforts will provide information on additional potential social well-being constructs to include in a conceptual framework that describes how child and family social well-being may be influenced by participation in TANF and other employment and training-related programs. Through this GenIC, the data will be used to contribute to (but not alone address) the development of a conceptual framework about family social well-being that will use information from multiple information gathering activities (a literature and conceptual model scan and synthesis; input from government, academic, and other experts with lived experience), including focus groups with parents and caregivers with lived experience in employment and training programs. In addition, the research team will produce a brief for the parents and caregivers that participated in the focus groups to share back the themes they heard from them on social well-being and inequities the parents and caregivers encountered while participating in the programs.

This proposed GenIC meets the following goals of ACF’s generic clearance for formative data collections for research and evaluation (0970-0356):

• Inform the development of ACF research.

• Maintain a research agenda that is rigorous and relevant.

• Ensure that research products are as current as possible.

The information is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information. The information is not intended to be generalized and only provides information from and about the parents and caregivers with lived experience who participate in the focus groups.

Research Questions

The information collected through the parent and caregivers with lived experience focus groups is intended to address the following research questions:

1. What do a group of parents and caregivers with lived experience in employment and training programs consider as important for their general social well-being?

2. What do a group of parents and caregivers with lived experience in employment and training programs think shapes their social well-being?

3. From the perspective of a group of parents and caregivers with lived experience in employment and training programs, what are the experiences in TANF programs? What structural inequities affect these parents’ and caregivers’ experiences in the types of programs that are studied within OPRE’s Welfare and Family Self-Sufficiency Research Portfolio?

Study Design

The research team will conduct seven focus groups with parents and caregivers with lived experience who are ages 18 and older and who are participating or formerly participated in TANF or employment and training programs. For the purposes of the focus groups, the research team is defining parents and caregivers as adults who bring up and are responsible for a child. The research team recognizes that families are complex and not all caregivers are biological parents. For each of the focus groups, the parents and caregivers will share characteristics similar to other members of the same group (e.g., parents of children in middle school). The research team plans for all data collection to occur virtually (i.e., via videoconferencing) for five focus groups and in-person for two focus groups. As part of the focus groups, we will collect demographic information from participants including gender identity, age, race and ethnicity, and the number and age of children in a family. This information will be used to document the backgrounds of participants in a final descriptive brief that describes what families consider important to their well-being.

This approach is intended to surface a diverse set of parent and caregiver perspectives on social well-being. The research team will document how parents and caregivers define social well-being, and any inequities they encounter as part of their participation in the programs. While the study aims to reflect a diverse set of perspectives, only a small number of focus groups will be conducted and therefore cannot account for the full range of participant background characteristics. The results are not designed to be representative of or generalizable to all parents and caregivers who participate in programs or interventions aimed to increase economic security, rather these groups will capture the experiences of the specific parents and caregivers who participated in the focus groups.

Limitations to the findings, such as generalizability, will be noted in the summary descriptive brief, as appropriate.

Instrument	Respondent, Content, Purpose of Collection	Mode and Duration
Instrument 1: Focus group discussion guide	Respondents: Up to eight parents or caregivers in each of seven focus groups. Content: Focus group participant information including gender identity, age, ethnicity, race, number of children and age of children. Parents’ and caregivers’ definition of what it means to be doing well. Families’ experiences during and after participation in TANF or an employment or education and training program. Inequities parents and caregivers face as part of their participation in the program. Parents’ and caregivers’ participation in a program as it relates to: interacting with their family in a healthier and more supportive way; feeling physically, mentally, and emotionally healthier; being connected and safe in their communities; and having a sense of belonging, inclusion, and respect. Purpose: To learn about how clients participating in programs and interventions that aim to increase economic security define social well-being and inequities they may face.	Mode: In-person and virtual focus groups Duration: 90 minutes per group

Other Data Sources and Uses of Information

There are no other data sources for this data collection. This data collection will inform the development of the conceptual framework, which is referenced above.

A3. Use of Information Technology to Reduce Burden

The proposed parent and caregiver focus groups will be conducted either in-person or virtually through Zoom. The use of technology in this case is intended to both reduce burden on respondents and address potential participant concerns related to COVID-19. The research team will audio-record focus groups, with the permission of the respondents, in order to assist with written note-taking.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

Based on consultations with experts and federal staff, we are not aware of any existing data that would meet the goals of this GenIC. The focus groups we propose conducting will collect information that is not available from any other data source. While other sources of information across the project will inform the conceptual framework, no other sources currently exist that will provide the information we will collect from the focus groups (families’ perspectives on their well-being as it relates to participation in TANF and related programs).

A5. Impact on Small Businesses

No small businesses will be involved with this GenIC.

A6. Consequences of Less Frequent Collection

This is a one-time data collection.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published two notices in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This first notice was published on November 3, 2020, Volume 85, Number 213, page 69627, and provided a sixty-day period for public comment. The second notice published on January 11, 2021, Volume 86, Number 6, page 1978, and provided a thirty-day period for public comment. ACF did not receive any substantive comments.

Consultation with Experts Outside of the Study

The study consulted with several outside experts on the research questions and study design. One expert reviewed the focus group discussion guide. There are two parents on our expert group and we asked one parent, who was a former TANF program participant, to provide input and make suggestions. The experts are listed in Table A8.1.

Table A8.1. List of Experts Consulted

Name	Affiliation
Danielle Bohannan	Colorado Family Voice Council
Natasha Cabrera	University of Maryland, College Park
Kathryn Edin	Princeton University
Heather Hill	University of Washington
Tyreese Nicolas	Massachusetts Department of Transitional Assistance
Ellen Pinderhughes	Tufts University
Babette Roberts	Washington State Department of Social and Health Services
Marjorie Sims	Ascend at the Aspen Institute
Tanisha Tate Woodson	Education Northwest
Ealasha Vaughner	Clayton Early Learning Policy Council and Colorado Family Voice Council

A9. Tokens of Appreciation

The focus groups are intended to gather input about parents’ and caregivers’ lived experiences. Without their valuable input, the research team would be unable to answer the study’s research questions and provide important information to ACF to inform future work. A token of appreciation signals to potential respondents their lived experience is valued and community members are respected (Chicago Beyond 2019).

Focus group data are not intended to be representative in a statistical sense, in that they will not be used to make statements about how all people who participate in programs or interventions that aim to increase economic security define social well-being. However, it is important to secure participants with a range of background characteristics to capture a variety of possible experiences. All participants will be individuals who are participating or formerly participated in state or county level TANF programs aimed at increasing economic security. Tokens of appreciation will be provided to offset the direct costs incurred by respondents for attending the focus groups, such as arranging child care or transportation for those in-person. This will help ensure a broader range of participant backgrounds and experiences as participants will not have to overcome financial barriers to participate in the study. Participants in the 90-minute virtual focus groups will receive a $50 gift card to account for incidental expenses such as child care that may otherwise prevent their participation. Participants who attend the 90-minute focus group in person will receive a $65 gift card to account for incidental expenses such as transportation or child care that may otherwise prevent their participation.

The amount proposed is also based on research on tokens of appreciation and the research team’s experiences with other federal studies. Research has shown that a token of appreciation can be an effective way to increase study participation among populations including individuals from that have low incomes or households that have relatively lower levels of education, which are demographics of interest in the Measuring SUCCESS project. Specifically, Bierer et al. (2021) and Cheff and Roche (2018) report that underpaying research participants can jeopardize efforts to be inclusive in studies, as income status can be a barrier to participation (Bierer et al. 2021, Cheff and Roche, 2018). Offering a token of appreciation of $50 in qualitative research such as the proposed focus groups is also supported by research. A nationally representative study of U.S. adults comparing how different token amounts affect willingness to participate in qualitative research found that participants offered a $50 token of appreciation were more likely to participate in research than those who were offered smaller tokens of appreciation (Kelly et al. 2017). Based on the research, the population we are trying to reach, and experience of the research team, we believe $50 is an appropriate amount for the time and cost associated with participation in virtual focus groups but is not so high as to appear coercive for potential participants. We also feel that the additional $15 for in-person participation is necessary to offset the costs of transportation.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

The research team will collect the respondent's name, email address, phone number, gender identity, age, race and ethnicity, and number and age of children. The purpose of collecting this PII is to:

• Contact potential focus group participants via phone or email² to recruit and provide text message reminders to boost participation.

• Provide information on the demographics of the focus group participants so that the summary descriptive brief can describe the types of families that were involved in the focus groups.

All notes taken during the focus groups will be de-identified (see below for additional information). Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the contractor will comply with all Federal and Departmental regulations for private information.

Data Security and Monitoring

As specified in the contractor’s contract, the contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ Personally Identifiable Information (PII). The contractor will ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor who perform work under this contract/subcontract are trained on data privacy issues and comply with the above requirements. All study staff with access to PII will receive study-specific training on (1) limitations on disclosure; (2) safeguarding the physical work environment; (3) storing, transmitting, and destroying data securely; and (4) reporting on potential data breaches or incidents that adhere to HHS requirements. The research team conducts additional trainings for staff that are responsible for data that are relevant to their specific data collection roles. For example, focus group facilitators are also trained on ensuring that participants will be able to answer questions in a safe and secure manner. These procedures will be documented in training manuals. Refresher training will occur annually.

Notes taken during the focus groups will be kept on the contractor’s internal shared drive with access restricted to the research team.³ The contractor’s research team will de-identify notes by substituting numerical codes for interviewee names as soon as feasible. Interviews may be recorded on digital audio recorders. The contractor’s staff will upload the recordings to a transcription vendor via their secure upload process via Box and retrieve the completed transcriptions and download them to a restricted drive on the contractor’s secure server. Once the contractor’s staff confirm that a transcription has been retrieved, the recording will be deleted from the digital audio recorder using a cleaner program (for example, Eraser). Staff conducting these interviews will be trained on doing so in a safe and secure manner, recording and uploading the interviews, and retrieving them securely.

As specified in the contract, the contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The contractor will securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Information Processing Standard. The contractor will ensure that they incorporate this standard into their property management/control system and establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology requirements and other applicable Federal and departmental regulations.

The contractor will dispose of the primary data and files containing PII or other confidential data created during the course of the study. The contractor will retain the data at least through the end of the project's contract. The contractor shall destroy data at the end of the contract in accordance with specifications provided by ACF.

A11. Sensitive Information ⁴

Though it won’t be explicitly prompted, the research team may collect sensitive information related to gender identity, receipt of economic assistance, or immigration/citizenship status if respondents volunteer such information as part of the focus group discussions. We are asking this demographic information to be able to describe the types of families that provided input within the brief that we will develop. Although uncovering this type of information is not the focus of the study, it may be relevant to the types of services that focus group respondents receive from the programs or interventions aimed at increasing their economic security. It is also possible that respondents may speak critically of partners, services, or programs during conversations. Questions about their experiences are necessary to learn about when informing a broader understanding of how they define social well-being. Given the open-ended nature of focus group questions, respondents may volunteer other sensitive information, although no other sensitive information is explicitly prompted. Data will be handled and monitored securely and notes de-identified, as described above.

This study will be submitted to the contractor’s Institutional Review Board (IRB), HLM IRB. Review will be finalized by the time data collection begins.

A12. Burden

Explanation of Burden Estimates

The research team plans to conduct seven focus groups with up to eight parents or caregivers. The focus groups will take place over 90 minutes. We have estimated burden related to the GenIC to be 2 hours, to account for the time to prepare to learn about the focus groups and to correspond with the research team about the time and location of the groups.

Estimated Annualized Cost to Respondents

We estimate respondents hourly wage as the Federal minimum wage of $7.25 (U.S. Department of Labor).

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total/Annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Focus group discussion guide	56	1	2	112	$7.25	$812

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total/annual cost for the data collection activities under this request will be $131,713. This includes both direct and indirect costs.

Cost Category	Estimated Costs
Field Work	$75,841
Analysis	$30,000
Summary Descriptive Brief	$25,872
Total costs over the request period	$131,713

A15. Reasons for changes in burden

This is for an individual GenICunder the umbrella formative generic clearance for ACF research (0970-0356).

A16. Timeline

Activity	Date
Focus groups	Scheduled one month after OMB approval, and all focus groups will be completed within three months
Data Analysis Completed	One month after focus groups are complete
Brief	Nine months after focus groups

A17. Exceptions

No exceptions are necessary for this GenIC.

Attachments

Instrument 1: Focus group discussion guide

Appendix A: Focus group consent form

Appendix B: Outreach email to focus group participants

Appendix C: Outreach email 2 to focus group participants

Appendix D: Outreach email confirmation to focus group participants

Appendix E: Reminder email to focus group participants

Appendix F: Focus group FAQs

Appendix G: Outreach email to potential participants who contact the study

References

Bierer Barbara E, Sarah A White, Luke Gelinas, and David H Strauss. (2021). Fair payment and just benefits to enhance diversity in clinical research. Journal of Clinical and Translational Science 5:1,e159.

Cheff, Rebecca and Brenda Roche. (2018). Considerations for Compensating Research Participants Fairly and Equitably: A Think Piece. Toronto: Wellesley Institute.

Chicago Beyond. “Why Am I Always Being Researched?” (2019). Equity Series, Volume One. Chicago: Chicago Beyond.

Kelly, Bridget, Marjorie Margolis, Lauren McCormack, Patricia A. LeBaron, and Dhuly Chowdhury. (2017). What Affects People’s Willingness to Participate in Qualitative Research? An Experimental Comparison of Five Incentives. Field Methods, 29:4, 333-350.

U.S. Department of Labor. Federal Minimum Wage Rate under the Federal Fair Labor Standards Act. https://www.dol.gov/general/topic/wages/minimumwage.

1 The project’s working definition of family social well-being considers whether all members of the family: (1) are physically and mentally healthy; (2) are safe and supported in the community; (3) are developing cognitively and socially; (4) have a sense of belonging, inclusion, and respect based on their identity as an individual and role in the family; and (5) are interacting within their family in a healthy and supportive way.

2 We will be reaching out to up to 9 state or county level TANF administrators to identify participants. This is not subject to PRA.

3 Data on Mathematica’s internal shared drive are protected using FIPS 140-2 compliant encryption and the network is protected by a Cisco firewall. Access to the drive must be approved by project team leadership.

4 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., Unemployment Insurance or WIC or SNAP); immigration/citizenship status.

11