Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
The Experiences, Needs, and Voices of Workers in Low-Income Households During the COVID-19 Public Health Emergency
Formative Data Collections for ACF Research
0970 – 0356
Supporting Statement
Part A
July 2020
Submitted By:
Office of Planning, Research, and Evaluation (OPRE)
Administration for Children and Families (ACF)
U.S. Department of Health and Human Services (HHS)
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officer: Hilary Bruck
A1. Necessity for Collection 4
A3. Use of Information Technology to Reduce Burden…..…………………………………………………………….6
A5. Impact on Small Businesses 6
A6. Consequences of Less Frequent Collection 7
A7. Now subsumed under 2(b) above and 10 (below) 7
A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing 8
A12. Burden 10
A13. Costs 10
A14. Estimated Annualized Costs to the Federal Government 10
A15. Reasons for changes in burden 10
Attachments
Instruments
Instrument 1. In-depth discussion guide
Appendices
Appendix A. Study flyer
Type of Request: This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for ACF Research (OMB #0970-0356).
Description of Request: The Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) is proposing to conduct in-depth discussions with 44 participants from four programs that are being evaluated in one of two ongoing studies of employment programs for people with low income—(1) Evaluation of Employment Coaching for TANF and Related Populations (Evaluation of Employment Coaching, OMB #0970-0506) and (2) Next Generation of Enhanced Employment Strategies Project (NextGen Project, OMB #0970-0545). The overall goal of the information collection is to understand, from the perspective of participants, how programs designed to support low-wage workers have responded to the 2019 Novel Coronavirus (COVID-19) public health emergency (PHE) and whether and how they are meeting the needs of participants.
This information will support the conduct of the aforementioned evaluations, to ensure they are relevant and account for current conditions. It will also inform ACF’s research agenda more broadly, offering considerations for supporting the well-being of families particularly vulnerable to the economic effects of COVID-19.
The data collected in the study are not intended to be generalized to a broader universe. We do not intend for this information to be used as the principal basis for public policy decisions.
Time Sensitivity: We would like to start data collection as soon as possible. It is critical that this information be collected in the short-term in order to best capture respondents’ early and evolving COVID-19 related experiences and to provide programs with timely information that they can use to be more responsive in meeting the needs of participants.
The COVID-19 Public Health Emergency (PHE) is causing major and far-reaching disruptions for many workers and their families. Millions of people have been laid off, furloughed, had their hours reduced, or quit their job as a result of COVID-19. Others have faced the challenges of working with children at home as schools and child-care facilities have closed. Still others have had to work in conditions that potentially expose them to COVID-19. Because of the industries and occupations they work in, workers in low-wage jobs are more likely than workers in higher-wage jobs to be affected by these disruptions.1 Yet, workers in low-income households have fewer resources to weather these disruptions.
Employers are being challenged to protect their workers while maintaining their short- and long-term viability. The choices and trade-offs employers make in the face of this crisis affect their workers. Similarly, the public and private organizations that make up the safety net are being challenged to respond to an increased need for supports while also having to devise creative ways to meet those needs amidst mandates and calls for social distancing and closures. The gradual lifting of restrictions presents additional challenges and complexities for employers, safety net programs, and workers.
The perspectives of the workers themselves are essential to developing thoughtful and meaningful policy and programmatic solutions that meet their needs. Literature on conducting high-quality qualitative research suggests that often, “program participants are the best source of information on their own conditions.”2 Data from this information collection will offer a critical perspective on low-wage workers’ experiences and needs during the COVID-19 PHE, how programs designed to support low-wage workers are meeting the needs of participants during the PHE, and what else is needed to help ensure the well-being of families particularly vulnerable to the economic effects of COVID-19.
There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at the discretion of the agency.
Purpose and Use
The findings from this study will serve two key purposes. First, the findings will inform the conduct of the Evaluation of Employment Coaching and the NextGen Project to ensure they are relevant and account for current conditions under the PHE, to fully understand the implementation of the programs included in these ongoing studies, and to inform the provision of technical assistance to these programs during the evaluation period in order to strengthen their service delivery and better equip them to respond to the ongoing and evolving PHE. This will help ensure that the programs being evaluated provide a strong test of effectiveness for the evaluations. Second, the findings will inform ACF’s research agenda more broadly by providing timely and useful insights and lessons about how to meet low-wage workers’ needs during an ongoing or future PHE.
inform the development of ACF research
maintain a research agenda that is rigorous and relevant
The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.
Guiding Questions
Five broad questions guide this study:
How did the COVID-19 PHE affect participants’ employment?
How did participants respond to changes in their employment situation? What coping strategies did they use, what supports did they access, and what are their plans and hopes for future employment?
How, from participants’ perspectives, did their employers respond to the PHE and disruptions to business? How did employers communicate and manage changes to employees’ status, work responsibilities, and work environment, and how did employers assist workers in responding to any changes in their employment?
How, from participants’ perspectives, did the programs participating in the NextGen Project and Evaluation of Employment Coaching engage with participants and respond to their needs? In what ways did programs change the supports they offer?
What were workers’ unmet needs? In what ways could programs fill those needs?
Study Design
The proposed information collection will conduct in-depth discussions with 44 participants from four programs that are being evaluated in one of two ongoing OPRE studies of employment programs for people with low income: (1) the Evaluation of Employment Coaching (OMB #0970-0506) and (2) the NextGen Project (OMB #0970-0545). Mathematica is leading both studies under contract to OPRE. Other components of these ongoing studies will provide an analysis of program impacts based on a rigorous experimental study design and an implementation study based on implementation science principles. The proposed information collection will recruit and conduct the in-depth discussions with current program participants who are between 18 and 59 years old (i.e., working age), were working in February or March when the pandemic hit their area and continued to work or were laid off, furloughed, had their work hours reduced or quit as a result of COVID-19. Each discussion will last about 120 minutes. Study team members will use an in-depth discussion guide (Instrument 1. In-depth discussion guide) to facilitate the discussion. Additional details about participant recruitment are presented in Supporting Statement B, section B4.
The in-depth discussions addressed in this information collection request will further inform these two ongoing evaluations by providing timely information on working participants’ needs during a period of significant economic upheaval and implications of their experiences for these programs’ design and operations. Obtaining information and insights from the participants themselves provides an important perspective and authentic narrative that cannot be obtained through close-ended participant surveys. The study is not intended to provide results that are representative of program participants or generalizable to the broader population of participants in programs designed to support low-wage workers. Rather, the study will provide information on a range of experiences that program participants faced during the PHE.
Table 1. Study design
Instrument(s) |
Respondent, Content, Purpose of Collection |
Mode and Duration |
Instrument 1. In-depth discussion guide |
Respondents: 44
Content: Changes in employment related to the PHE, employer and program responses, and unmet needs
Purpose: To gather descriptive information on participant experiences with the PHE to inform the conduct of the evaluations and provision of technical assistance |
Mode: Webex audio and video conferencing
Duration: 120 minutes |
Other Data Sources and Uses of Information
The data from this information collection will be combined with data that will be collected under other components of the NextGen Project (OMB #0970-0545) and the Evaluation of Employment Coaching (OMB #0970-0506) to provide a fuller description of programs’ responses to the PHE. Additional information regarding the analysis and use of the data from this information collection in combination with other data sources is presented in Supporting Statement B, section B7.
The study team plans to use information technology to reduce burden on respondents wherever possible. Information about the study (Appendix A. Study flyer) will be emailed to potential respondents during the recruitment phase, as possible. The in-depth discussions will be conducted by teleconference or Webex according to each respondent’s preference.
The data that will be collected via in-depth discussions with program participants are not available in a comparable format elsewhere and are not being gathered through other sources.
No small businesses will be involved with this information collection.
This is a one-time data collection.
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. During the notice and comment periods, no substantive comments were received.
The study team consulted with the following individual on the design of the in-depth discussion guide:
Tanisha Tate Woodson, Ph.D.
NPC Research (Northwest Professional Consortium, Inc.)
975 SE Sandy Blvd.,
Ste. 220
Portland, OR 97214
Phone: (503) 243- 2436 x 118
Email: woodson@npcresearch.com
Program participant discussion data are not intended to be representative in a statistical sense, in that they will not be used to make statements about the prevalence of experiences in the program population. However, within each selected program participating in the NextGen Project or Evaluation of Employment Coaching, it is important to speak with participants with a range of characteristics and experiences to capture a variety of perspectives on the programs’ responses to the PHE and the range of needs among participants. Without offsetting the direct costs incurred by respondents for participating in discussions, such as arranging child care, the study team increases the risk that only those individuals who can overcome the financial barriers or who are not currently working will participate in the study, affecting the quality and completeness of the resulting data and insights.
To offset costs of participation in the 120-minute interviews, the study team will offer program participants a $60 gift card to Walmart, Target, or Amazon. An explanation of this token of appreciation will be provided during the initial outreach call to program participants, and again at the beginning of the scheduled interview. This token of appreciation is modeled on another ACF study entitled Parents and Children Together (PACT). Respondents (who were low-income fathers) received a $60 gift card for a 2-hour in-depth interview (OMB #0970-0403). The PACT study observed overall response rates of 88 and 72 percent for the two waves of interviews, respectively.
The token of appreciation will be emailed to those with access to email upon completion of the discussion; gift cards will be mailed to respondents without access to email.
Personally Identifiable Information
Individuals’ names and telephone numbers will be collected during study recruitment for use in scheduling and conducting the in-depth discussions. The toll-free phone line available to potential respondents to express their interest in participating in the study and for scheduling interviews will be set up with a secure voice mailbox. Phone calls will be directed only to designated project staff who are responsible for scheduling the calls. If the phone goes unanswered, it will be diverted to the secure voice mailbox where interested participants will be asked to leave their name and phone number. The messages are password protected and interview schedulers will log messages in a file on the project’s secure drive and then delete it from voicemail. Messages are overwritten three times when they are deleted and cannot be forwarded or converted into emails. Information will not be maintained in a paper or electronic system from which it is actually or directly retrieved by an individual’s personal identifier.
Assurances of Privacy
Mathematica will protect respondents’ privacy to the extent permitted by law and will comply with all Federal and departmental regulations for private information. Mathematica has developed a data safety and monitoring plan that assesses all protections of respondents’ PII. Mathematica will ensure that its staff who perform work on this information collection are trained on data privacy issues and procedures and comply with the above requirements. All study staff with access to PII will receive study-specific training on (1) limitations on disclosure; (2) safeguarding the physical work environment; and (3) storing, transmitting, and destroying data securely.
Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. With consent from respondents, discussions will be audio recorded and transcribed to facilitate analysis. Discussion transcripts will not be shared by the study team with ACF or anyone else outside of the study team, except as required by law.
The study team sought and received Institutional Review Board (IRB) expedited approval for this information collection from the Health Media Lab IRB on June 30, 2020.
Data Security and Monitoring
All study team members have received training in data security procedures. The study team will store data collected in a restricted folder on Mathematica’s servers. Mathematica’s servers are located behind Mathematica’s firewall and housed in a locked data center located in Mathematica’s locked access-controlled office suite. Data are mirrored in a secure, fault-tolerant data center; only authorized Mathematica Information Technology Services staff have physical and or logical access to the data mirror. Sensitive data reside on a project-specific folder that is only accessible to Mathematica staff who have a business need-to-know, as restricted by identity-based policies and access control lists. The data are encrypted as they are stored on the server with an Advanced Encryption Standard (AES) 256-bit key, which is FIPS 140-2 compliant. The folder in which the data reside is backed up onto encrypted disks. These backups are overwritten every two months by backups of newer secure data, a process that enables compliance with secure data destruction requirements.
Information shared for transcription of audio recorded discussions will be sent via a secure file transfer protocol.
Some sensitive questions are necessary in a study that examines the effects of economic disruption due to COVID-19 on workers’ employment and their ability to support their families, and how programs addressed their needs. Before starting the in-depth discussions, all respondents will be informed that their identities will be kept private, that their participation is voluntary, that their responses will not affect any services or benefits they or their family members receive, and that they do not have to answer any questions that make them uncomfortable. Although some questions may be sensitive for some respondents, they have been successfully asked of similar respondents in other data collection efforts, such as Parents and Children Together (OMB #0970-0403) and the in-depth interviews already conducted for the Evaluation of Employment Coaching (OMB #0970-0506).
The sensitive questions in the data collection instruments relevant for this ICR include:
Income and fringe benefits. It is necessary to ask about any recent changes in earnings and fringe benefits, as a key goal of the study is to understand the effect of economic disruptions related to COVID-19 on participants’ income and well-being. During the in-depth discussions, the study team will ask about participants’ jobs, how their jobs were affected by COVID-19, and the impact of changes on their economic well-being.
Challenges to employment. It is important to ask about challenges to employment to understand how the programs might be supporting participants and their potentially unmet needs. Challenges discussed during the in-depth discussions may include working conditions, school closures and child-care needs, and transportation challenges.
Economic hardships. The in-depth discussions include questions about economic hardships, such as difficulty paying bills, credit problems, or needing to borrow money from friends. It is important to ask about these issues to understand program participants’ potentially unmet needs and how programs might address them.
Health. The in-depth discussions will ask about the health of the participant and their family members and whether health issues impact participants’ ability to work or sustain work. Health factors—both related and unrelated to COVID-19—could play a major role in participants’ ability to maintain employment.
Explanation of Burden Estimates
The total annual burden requested under this generic information collection is 88 hours (Table 2). This includes the time it will take to conduct each interview, which is 2 hours each for 44 respondents.
Estimated Annualized Cost to Respondents
The total annual cost for the data collection activities under this current request will be $638 (Table 2). The total estimated cost figures are computed from the total annual burden hours and an average hourly wage for participants. The average hourly wage of study participants is estimated to be $7.25, the 2020 federal minimum wage (Table 2).
Table 2. Annual burden hours requested under this information collection
Instrument |
No. of respondents (total over request period) |
No. of responses per respondent (total over request period) |
Avg. burden per response (in hours) |
Total/ Annual burden (in hours) |
Average hourly wage |
Total annual cost |
Instrument 1. In-depth discussion guide |
44 |
1 |
2.0 |
88 |
$7.25 |
$638 |
Estimated annual burden |
88 |
$7.25 |
$638 |
|||
There are no additional costs to respondents.
The total cost for the information collection under this current request will be $334,109 (Table 3). These estimates of costs are derived from Mathematica’s budgeted estimates and include labor rates, direct costs, and tokens of appreciation for respondents. There are no costs beyond the normal labor costs for staff.
Table 3. Estimated Annualized Costs to the Federal Government
Cost Category |
Estimated Costs |
Instrument Development and OMB Clearance |
$37,594 |
Field Work |
$134,820 |
Publications/Dissemination |
$161,635 |
Total/annual costs |
$334,109 |
This is for an individual information collection under the umbrella formative generic clearance for ACF research (OMB #0970-0356).
Schedule
Task Completion date__________
Recruitment, data collection, transcription 6 weeks after OMB approval
Code and analyze interview data 10 weeks after OMB approval
Produce draft report 18 weeks after OMB approval
Revised report #1 24 weeks after OMB approval
Revised report #2 30 weeks after OMB approval
Final report 36 weeks after OMB approval
No exceptions are necessary for this information collection.
Attachments
Instrument 1. In-depth discussion guide
Appendices
Appendix A. Study flyer
2 Posavac, Emil J. Program Evaluation: Methods and Case Studies. Routledge: New York, New York. 2016.
3 Examples of sensitive topics include (but are not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Naomi Goldstein |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |