Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Testing Identified Elements for Success in Fatherhood Programs (Fatherhood TIES)
OMB Information Collection Request
0970-0622
Supporting Statement
Part B
Type of Request: Revision
July 2024
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Katie Pahigiannis
Rebecca Hjelm
B1. Objectives
Study Objectives
The Testing Identified Elements for Success in Fatherhood Programs (Fatherhood TIES) project is testing the “core components” of fatherhood programs. Core components are the essential functions, principles and elements that are judged as being necessary to produce positive outcomes. The project is engaging five Fatherhood–Family-Focused, Interconnected, Resilient and Essential (Fatherhood FIRE) grant recipient organizations to implement and test one of three core components: targeted program content (parenting); systems navigation (e.g., family court, child support, child welfare); and goal setting and engagement supports to help the father to achieve his goals. Targeted program content will be implemented by one Fatherhood FIRE grant recipient; the other two interventions will be implemented by two Fatherhood FIRE grant recipients each.
Fatherhood TIES will enroll up to 3,000 fathers across the grant recipient organizations to participate in the study. Using the information gathered across several different data sources, the evaluation will include an implementation study and an impact study. The implementation study will describe who participated in fatherhood program services, how services operated, what fathers thought about services, and the challenges that program staff faced when implementing TIES services. The implementation study will provide lessons for the field on key elements for successful program implementation and barriers to overcome when implementing these core components.
The impact study will use experimental research methods to rigorously evaluate whether promising core components can bring about positive outcomes for fathers and their families. The specific outcomes of interest include employment and earnings, father-child relationship quality and co-parenting relationship quality.
Fatherhood TIES includes two phases of data collection. Supporting Statement A and B are formatted to describe Phase 1 data collection, which has already been approved under this OMB number (0970-0622) and is currently in process, and Phase 2 data collection, for which we are currently submitting for review and approval.
Phase 1 (Approved and In-Process; No changes from previously approved justification)
The Phase 1 approval covered consent of participants to enter the study, baseline data collection, and obtaining information about program processes and outcomes during the study period to support the study team’s understanding of implementation and how it could be improved.
Phase 2 (Current Request)
This current Phase 2 request includes additional data collection efforts such as the nine-month follow-up survey, semi-structured interviews with program staff and co-parents, father focus groups, and participatory research methods of photo voice and audio journaling.
Generalizability of Results
Phase 1 and Phase 2 (No changes from previously approved justification)
Overall, Fatherhood TIES is designed not to promote statistical generalization to other sites or service populations, but to be internally valid for the grant recipient organizations involved in the study.
This study will include an implementation study for each of the five Fatherhood TIES grant recipient organizations. The implementation study is intended to present an internally valid description of the service population and the implementation of the interventions in chosen Fatherhood TIES grant recipient organizations, not to promote statistical generalization to other sites or service populations.
Fatherhood TIES will include randomized controlled trials involving each of the five Fatherhood TIES grant recipient organizations. The randomized study is intended to produce internally valid estimates of the intervention’s causal impact, not to promote statistical generalization to other sites or service populations.
Appropriateness of Study Design and Methods for Planned Uses
Phase 1 and Phase 2 (Approved and In-Process; Updates from previously approved justification)
Conducting both an impact and implementation study will allow the team to meet its goals of testing the effectiveness of core components in producing positive outcomes for fathers who engage in core component interventions. The impact study will include a distinct service contrast to determine the effectiveness of the core components.
The primary sources for the impact analysis are the baseline survey (Instrument #1) and a nine-month follow up survey (Instrument #11). The team explored the possibility of requesting National Directory of New Hires (NDNH) data, but after approval of Phase 1, the Fatherhood TIES study team decided not to pursue NDNH data and therefore is not collecting Social Security Numbers for individuals agreeing to participate in the study, as originally proposed. Fatherhood TIES participation outcomes will be measured using existing data from nFORM, a data collection system required of all Responsible Fatherhood grant recipient organizations (OMB #0970-0566). Program staff record information about father participation in services in nFORM; it is a standardized source of participation information that can serve as context to the analysis of other outcomes and qualitative implementation study data collection.
The implementation study will provide useful feedback from both fathers and program staff on the core components and how they were implemented. Methods for collecting information from staff and fathers include reflection forms (Instruments #3 and #4) and semi-structured interviews (Instruments #5 and #7).
As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
Phase 2 (Current Request)
This information collection request for Phase 2 includes additional data collection activities associated with the Fatherhood TIES implementation and impact studies. For implementation research, this includes:
Instrument #6 Staff Interview
Instrument #7 Co-Parent Interview
Instrument #8 Father Focus Group
Instrument #9 Photo Voice
Instrument #10 Audio Journaling
Appendix #4 Photo Voice Training Slides
Appendix #5 Audio Journaling Training Slides
Appendix #6 Father Focus Groups Training Slides
Appendix #7 Photo Voice Training Guide
Appendix #8 Audio Journaling Training Guide
Appendix #9 Father Focus Groups Training Guide
For the impact study this includes the nine-month follow-up survey (Instrument #11).
Implementation study instruments are outlined further in Section B4. Data Collection Activities. These data collection activities involve semi-structured interviews of staff (Instrument #6) and co-parents (Instrument #7) as well as focus groups with fathers (Instrument #8), to assess the implementation of core components at Fatherhood TIES program sites. Audio journaling (Instrument #10) and photo voice (Instrument #9) are participatory research methods which aim to enroll up to 60 fathers in total.
The appendixes in this submission are training slides for photo voice (Appendix #4), audio journaling (Appendix #5), and father focus group facilitation (Appendix #6). Training slides relate to human subjects’ research protocols as well as the specific structure of each method (photo voice, audio journaling, and father focus groups). Training guides that accompany the training slides are attached as Appendix #7 (photo voice), Appendix #8 (audio journaling), and Appendix #9 (father focus groups). These appendixes are described in greater detail in Supporting Statement A.
B2. Methods and Design
Target Population
Phase 1 and Phase 2 (Approved and In-Process; No changes from previously approved justification)
The target population for Fatherhood TIES are fathers. Specifically, those who are eligible for Fatherhood FIRE grant recipient organization services. These individuals must be 18 years or older and have children expected or up to age 24 years old. They could be biological fathers who identify as married or unmarried, expectant fathers, adoptive fathers, stepfathers or individuals who live with children who are acknowledged as father figures (i.e., grandfathers or foster fathers). Fathers could be low-income, non-custodial or custodial single fathers, receiving TANF, previously received or eligible to receive TANF, or participating in Head Start or Healthy Start programs. Fathers may also have had a history of involvement with the criminal justice system or speak English as a second language.
Site Selection
Phase 1 and Phase 2 (Approved and In-Process; No changes from previously approved justification)
Fatherhood TIES used a mix of existing data sources and individualized discussions with nine Fatherhood FIRE grant recipient organizations to select study sites. First, the study team used existing Fatherhood FIRE grant recipient organization enrollment data from nFORM to identify the top 25 programs enrolling the largest number of fathers annually. Then, the team reviewed additional information from nFORM such as program descriptions and aggregate data describing participant demographics, program content, program mode, location, and implementation experiences. The combination of these two steps narrowed the pool of potential sites to nine organizations with the strongest fit for TIES.
Representatives from each of the nine organizations were invited into conversation with the study team. The team provided an overview of the proposed study, offered to answer questions, and guided staff through design brainstorming activities. The study team used these individualized discussions to recommend five grant recipient organizations as study sites. All five have agreed to participate in the Fatherhood TIES study.
B3. Design of Data Collection Instruments
Development of Data Collection Instruments
Phase 1 (Approved and In-Process; No changes from previously approved justification)
Fatherhood TIES study enrollment builds upon each fatherhood program’s existing data collection processes. This includes their use of the nFORM management information system and surveys that were developed for federal Fatherhood FIRE grant recipient organizations (under OMB #0970-0566). The baseline survey (Instrument #1), which takes about 22 minutes, is administered through a separate web-based system. The baseline survey is designed to collect details about fathers that are not already collected in nFORM for the purpose of improving the sensitivity of the impact analysis; the nine-month follow up survey (Instrument #11) uses many of the same questions asked at baseline.
The program information and management tool (Instrument #2, called the TIES Table) is based on previous program information and management tools used by the study team. The content noted in the headers is tailored to the need of each Fatherhood TIES study site.
The reflection for staff (Instrument #3) and reflection for fathers (Instrument #4) protocols are based on the study team’s experience with a similar approach in the Strengthening the Implementation of Responsible Fatherhood Programs (under OMB # 0970-0566). Instruments #3 and #4 were developed with input from Fatherhood FIRE grant recipient program staff and former fatherhood program participants.
Table B1 provides more information about these data collection instruments.
Table B1. Data Collection Instruments
Instrument |
Details of data collection |
nFORM |
|
Baseline survey (Instrument #1) |
|
Program information and management tool (Instrument #2) |
|
Reflections from staff (Instrument #3) |
|
Reflections from fathers (Instrument #4) |
|
The study team worked with each Fatherhood TIES grant recipient organization to fit study enrollment processes seamlessly into their existing program enrollment processes.
The baseline survey (Instrument #1), reflections from staff (Instrument #3), and reflections from fathers (Instrument #4) were each pre-tested with fewer than nine individuals. Overall, suggestions helped to remove unnecessary or redundant questions and to change the language of certain questions so that they were asked in more culturally responsible, understandable, and readable ways.
The baseline survey (Instrument #1) was pre-tested with four fathers. On average, the fathers reported that they understood the questions and the wording and that they could easily read and understand them. Feedback from the fathers prompted the study team to add text to clarify the instructions and intent of questions such as why the father is asked to only respond to questions about one child.
The reflection from fathers (Instrument #4) was pre-tested by two fathers. Based on the feedback, one key revision was reducing the number of questions asked so that fathers would be able to access and complete the reflection easily on devices like a mobile phone. Staff from three fatherhood programs also provided feedback about ways the reflection questions could be made more specific.
The reflection from staff (Instrument #3) was reviewed by three staff from three fatherhood programs. Changes made based on staff feedback reduced the number of questions and revised language to better align with the core components being tested.
Phase 2 (Current Request)
Instrument |
Details of data collection |
Staff Interviews (#6) |
|
Co-Parent Interviews (#7) |
|
Father Focus Groups (#8) |
|
Photo voice (#9) |
|
Audio Journaling (#10) |
|
Follow-up Survey (#11) |
|
The staff interview protocol (Instrument #6) was pre-tested by two staff members from two Fatherhood TIES program sites. Interviews were 1-hour each and took place virtually on Zoom. Overall, their suggestions helped to improve understandability and relatability. For example, questions were reorganized, and language was edited throughout to make it clearer that the questions intend to gather information specific to implementing the Fatherhood TIES intervention, rather than the larger organization within which the intervention operates.
Six fathers pretested the audio journaling, photovoice, father focus group, and co-parent interview protocols. This discussion took place virtually on Zoom and lasted around 60 minutes. The audio journaling, photovoice, and father focus group protocols were simultaneously pre-tested with pairs of fathers in Zoom breakout rooms. In general, their review provided guidance to the study team to clarify questions or prompts that were not clear, adjust language to be more accessible to fathers.
For the co-parent interview protocol (Instrument #7), one father shared that his co-parent felt like the general use of term “co-parent” to refer to the mother while using the term “father” diminished her role as a mother and suggested alternate phrasing. The protocol was revised to refer to both parents as co-parents.
For the father focus groups protocol (Instrument #8), feedback was primarily identifying language that was confusing or could be misunderstood by other fathers. Fathers also gave suggestions to make the wording of the questions sound more conversational.
For the photo voice protocol (Instrument #9), fathers found the questions to be appropriate and engaging. They provided some examples of photos they would take that will be incorporated into the photo voice training as examples to other fathers.
For the audio journaling protocol (Instrument #10) changes were made to the wording of two prompts to make them clearer.
Five fathers pretested the follow up survey through cognitive testing to identify problems with the questionnaire items and difficulties respondents could have in answering them. Both self-administration and interview-administration approaches were tested. Father feedback prompted the study team to make clarifications to the survey question wording (such as clarifying the list of possible people a child could live with or the time period of interest), adjusting skip patterns, and to provide additional context or instructions for some questions.
B4. Collection of Data and Quality Control
Recruitment Protocol
Phase 1 (Approved and In-Process; No changes from previously approved justification)
The Fatherhood TIES team integrated study enrollment into each of the five Fatherhood TIES grant recipient organizations’ recruitment and enrollment processes. First, the program staff confirm program eligibility in the nFORM system, checking that the participant is an adult and is a biological or expectant father, stepfather, or other father figure raising a child who is up to age 24. After identifying prospective program participants who are eligible for study participation, program staff administer the informed consent procedures and ask fathers to sign an electronic consent form within nFORM (Appendix #1). At two participating programs that are implementing an intervention to support fathers’ navigation of systems (e.g., child support, child welfare, family court), program staff confirm study eligibility by screening on participant interest in increasing time with the child and/or improving the relationships with their coparent(s). Site staff screen potential participants to better target the intervention to fathers with an interest in the support being provided. Study enrollment began in February 2024 and will last for approximately 21 months. The study team anticipates enrolling all eligible fathers who provide consent over the enrollment period to meet our sample size target of 3,000 fathers. Fathers who consent to be in the study will be randomly assigned into a program group or control group. Fathers in the program group will have access to the individualized core components being tested in addition to the Fatherhood FIRE grant recipient’s business-as-usual programming. Fathers in the control group will only have access to the business-as-usual programming.
The random assignment (RA) ratio differs by intervention and may change over time to address sample size and cost considerations.
The parent coaching intervention started with a 40 – 60 program – control ratio. It uses individual random assignment to either the TIES program group or the business-as-usual (control) group which is assigned at intake; random assignment is conducted within the nFORM system.
The systems navigation intervention started with a 30 – 70 program – control ratio. It uses individual random assignment to either the TIES program group or the business-as-usual (control) group which is assigned at intake; random assignment is conducted within the nFORM system.
The financial support and coaching intervention started with 30 – 70 program – control ratio. Fathers are randomly assigned at intake using individual random assignment (conducted within nFORM) to either receiving financial coaching or not. Then financial supports undergoes cohort-based random assignment (conducted by MDRC) shortly after each cohort starts their workshop sessions. The factorial random assignment design is depicted in Table 3.
Table 3. Four Group Random Assignment Design
|
Financial Supports Yes Cohort RA |
Financial Supports No Cohort RA |
Financial Coaching Yes Individual RA |
Group
1: |
Group
2: |
Financial Coaching No Individual RA |
Group
3: |
Group
4: |
Phase 2 (Current Request)
Of the estimated 3,000 study enrollees, we expect to engage 80 fathers in focus groups, 60 fathers in photo voice or audio journaling, and 3,000 in reflection over the study period. We expect to interview 4 co-parents. We expect to interview approximately 50 staff members (10 from each of program locations) and anticipate engaging 50 staff in reflection over the study period.
For photo voice and audio journaling, fathers enrolled in TIES programming will be able to volunteer for participation in one of these studies. Staff will share information about the opportunity to participate, including criteria for selecting a limited number of fathers. Photo voice participants will be limited to fathers with full or joint custody of their child/children, that have smart phones with a camera and internet/data access, and who indicate an interest in photography. The team will also work with program staff to further determine other ways to identify and select those interested in this creative, participatory method. Audio journaling will be limited to fathers that have a smart phone and internet/data access and who indicate an interest in journaling by making voice recordings. The team will similarly work with staff in identifying and selecting participants for this opportunity. For both approaches, staff will recommend fathers to participate and help coordinate an information and training session that will be facilitated by the research team.
For co-parent interviews, staff will share information about the opportunity to participate with the fathers, including criteria for selecting a limited number of co-parents to interview. Interviews will be limited to co-parents that report having regular engagement or communication with the father about their children. We will also work with staff to determine how to best identify and select co-parents for participation. Co-parents will be identified through father referral. Fathers will be given an opportunity to reach out to the co-parent if they would like to or pass on their contact information to staff to explain the opportunity to participate. If the co-parent expresses interest, staff then pass on the co-parent's contact information on to the research team who will contact them directly to schedule an interview.
Data Collection Activities
Phase 1 (Approved and In-Process; No changes from previously approved justification)
The fathers that agree to participate in the study are asked to complete the baseline survey (Instrument #1). This occurs within MDRC’s FedRAMP-certified cloud-based environment via Qualtrics, with access to the survey granted by the grant recipient organization program staff. If internet access is not available for any reason, the program staff have the ability to administer the survey on paper.
Program staff enter information about intended and actual services delivered in the program information and management tool (Instrument #2), referred to as the “TIES Table.” This is an Excel file that is customized to each program and intervention, to ensure that they are capturing and monitoring implementation of Fatherhood TIES-specific services.
Reflections from staff (Instrument #3) and reflections from fathers (Instrument #4) take place in the middle of each Fatherhood TIES grant recipient cohort. These short surveys will be administered using Qualtrics to collect perspectives about services delivered or participated in.
Phase 2 (Current Request)
The implementation study will collect qualitative and quantitative data. The core set of research questions will be consistent across sites, but some data collection activities will be tailored to specific sites. Primary data sources to be gathered from each program include semi-structured interviews (Instrument #6) and reflection surveys with program implementing program staff (Instrument #3), focus groups (Instrument #8) and reflection surveys with fathers (Instrument #4), programmatic data available in nFORM, and follow-up surveys. Data sources will be tailored to each site based on the core component intervention being tested, including: co-parent interview (Instrument #7), photo voice (Instrument #9), and audio journaling (Instrument #10).
Implementation Study Data Sources:
Staff interviews.
Implementing program staff will be asked to participate in semi-structured interviews (Instrument #6). Interviews will be conducted by the research team in person or by phone or video conference. Interviews will explore the context in which the core component intervention is being implemented, the intervention approach, implementation successes and challenges, perceived participant responses to the interventions, and opportunities for improvement.
Co-parent interviews.
Co-parents of fathers receiving the core components at select sites will be asked to participate in a semi-structured interviews (Instrument #7). Interviews will be conducted by the research team in-person or by phone or video conference. Interviews will assess co-parent perspectives on fathers’ involvement with children, parenting, and co-parenting as it relates to the core component intervention being tested.
Father focus groups.
A subset of fathers participating in the study will be asked to participate in focus group discussions (Instrument #8) that will be co-facilitated by the research team and Father Advisors (alumni fathers hired by program sites for the Fatherhood TIES study). Focus groups will be conducted in-person or by phone or video conference.
Photo voice.
Photo voice is a visual participatory action research tool. A subset of fathers at select sites will be trained and asked to take weekly photos in response to a research question prompt and then participate in a focus group discussion to collaboratively analyze the photos with the research team (Instrument #9). Over a three-week period, photos will be submitted through an electronic link using Qualtrics. After submitting photos, the research team will facilitate group or individual conversations with fathers to collaboratively discuss and analyze the photos. Findings will be discussed to identify themes.
Audio-recordings.
Audio journaling is an auditory participatory research approach in which fathers will be asked to make audio recordings in response to a weekly prompt about how they are applying knowledge and skills gained through their participation in the fatherhood program and its effect on their economic stability, experiences navigating systems, or on relationships with their children and co-parents. Audio recordings in response to weekly prompts (Instrument #10) will be made by a subset of fathers at select sites. Audio recordings will be made to a secure ZoomGov 1-800 line set up by MDRC. Recordings will be transcribed and analyzed to identify themes within and across sites/tests.
As an update to the Phase 1 submission, the study team has adjusted their thinking about the timing of reflections from staff (Instrument #3) and reflections from fathers (Instrument #4). Reflections from fathers (Instrument #4) will take place at the end of each Fatherhood TIES grant recipient cohort. Reflections from staff (Instrument #3) will take place every three months after study launch.
Impact Study Data Source:
Participating fathers will be asked to complete a follow-up survey (Instrument #11) about 9 months after program enrollment. Follow-up survey data collection is expected to occur from October 20241 to September 2026, based on Fatherhood TIES study enrollment beginning in February 2024. This survey will ask fathers to report on their receipt of fatherhood services and perceptions about involvement with their children, quality of father-child and co-parenting relationship, economic stability, and personal well-being.
Data Collection Monitoring
Phase 1 (Approved and In-Process; No changes from previously approved justification)
Throughout the study period, the Fatherhood TIES team meets regularly with participating grant recipient organizations to support and ensure high-quality study enrollment and data collection processes and procedures, including to review baseline characteristics (Instrument #1) of the program and control groups and review the data compiled in the program information and management tool (Instrument #2). The team meets with the program staff at a minimum once every two weeks. Additionally, the study team receives frequent data extracts from the nFORM system, which allows the Fatherhood TIES team to have near-real-time information on the frequency of missing data or other concerning trends in the data.
The study team also reviews information gleaned from reflections from staff (Instrument #3) and reflections from fathers (Instrument #4) at regular intervals to monitor program activities.
Phase 2 (Current Request)
The study team will review the findings from the father and staff reflections, staff interviews, co-parent interviews, father focus groups, audio journals, and photo analysis to monitor program activities throughout the study period. This ongoing review will help the study team understand how each core component is integrated within existing program services and will allow the study team to identify and address implementation challenges.
The implementation study team will coordinate data monitoring activities with the study team working most closely with each program in the study to share and reflect information collected with the program staff. This opportunity is designed to allow program staff to hear about what the research team is learning more frequently and reflect on that information together.
To monitor data collection for the 9-month follow up survey, Abt Global and MDRC will meet biweekly to track response rates and discuss successes and challenges with the data collection. As needed, communication will also occur via email and in ad hoc meetings. Additionally, Abt Global will provide a tracking report to MDRC monthly.
B5. Response Rates and Potential Nonresponse Bias
Response Rates
Phase 1 (Approved and In-Process; No changes from previously approved justification)
Phase 1 includes obtaining consent (Appendix #1) and baseline characteristics (Instrument #1) from fathers voluntarily seeking services at the five grant recipient organizations involved in the Fatherhood TIES study. Program staff inform eligible fathers of their invitation to be a part of the Fatherhood TIES study and let them know participation is entirely voluntary. There is not a response rate associated with consent or completion of the baseline survey. The consent process is not designed to produce statistically generalizable findings. Participation in the study is wholly at the respondent’s discretion. Response rates will not be calculated or reported.
The data collection activities associated with Instruments #2 through #4 are for the purpose of understanding how the Fatherhood TIES interventions are implemented. For the reflections for staff and fathers (Instruments #3 and #4), participation is wholly at the respondent’s discretion. Nonetheless, response rates will be calculated and reported for Instruments #3 and #4; response rates will not be calculated for Instrument #2. The value of fathers contributing to our understanding of fatherhood programming will be emphasized in all communication.
Phase 2 (Current Request)
The data collection activities associated with Instruments #5 through Instrument #10 are for the purpose of understanding how the Fatherhood TIES interventions are implemented. Request to participate in any of these activities is wholly at the respondent’s discretion. Response rates will not be calculated. The value of fathers and co-parents contributing to our understanding of fatherhood programming will be emphasized in all communication.
In a follow-up survey data collection effort for participants of fatherhood programs for the Building Bridges and Bonds (B3) project, Abt Global attained an 80 percent response rate. To obtain a similar response rate for the Fatherhood TIES follow-up survey (Instrument #11), the team will employ a comparable data collection approach (phone and in-person administration) which will be augmented with initial web administration to make it easier and more convenient for sample members to complete the survey on their own if preferred. In addition to making it easier for some sample members to respond, the inclusion of the web survey also brings cost efficiencies because it reduces the number of interviewer hours that are needed. Based on other projects that employ the web, phone, and in-person data collection approach, the team expects about 30% of the completed surveys to come by the web and the remaining to be split between phone and in-person administration.
Non Response
Phase 1 (Approved and In-Process; No changes from previously approved justification)
As participants will not be randomly sampled and findings are not intended to be representative, non-response bias (in the form of individuals declining participation in the study) will not be calculated. Respondent demographics will be documented and reported in written materials associated with the data collection. The team will caveat analysis as appropriate.
All efforts will be made to obtain consent for participation in Fatherhood TIES (Appendix #1) and the baseline survey (Instrument #1) for all eligible fatherhood program enrollees. The nine-month follow-up survey nonresponse is detailed below in the Phase 2 section.
The data collection associated with Instruments #2 through #4 will not be randomly sampled and findings are not intended to be representative, therefore non-response bias will not be calculated. The Fatherhood TIES team will emphasize the potential benefits of participating in data collection activities to increase motivation to participate; these benefits are not to the individual, per se, but to the fatherhood field broadly. To encourage fathers to provide insights about their experiences with the fatherhood programs (Instrument #4) the study team will provide each program with clear information about the data collection effort for staff to relay to fathers.
Phase 2 (Current Request)
The data collection associated with Instruments #5 through #10 will not be randomly sampled and findings are not intended to be representative, therefore non-response bias will not be calculated. The Fatherhood TIES team will emphasize the potential benefits of participating in data collection activities to increase motivation to participate; these benefits are not to the individual, per se, but to the fatherhood field broadly. To encourage fathers to provide insights about their experiences with the fatherhood programs the study team will provide each program with clear information about the data collection effort for staff to relay to fathers.
Abt Global will encourage response to the follow-up survey (Instrument #11) in multiple ways to mitigate nonresponse. Data collection will include multiple modes of contacting (both initial invitation and reminders) sample members (mail, email, text, phone calls, and in-person contacts) to reach sample members and encourage them to complete the survey. The contacts will feature compelling and motivating language to convey the importance of the survey and encourage sample members to respond. The messaging will also highlight the incentive that sample members will receive by completing the survey (which the research literature consistently shows encourages response).
The follow-up survey will also be able to be completed on the web, on the phone or in-person with an interviewer, which will allow sample members to complete the survey in their preferred mode that is easiest and most convenient for them. The modes will be offered sequentially to sample members. First, sample members will be invited to complete the survey on the web, which will be optimized so that it can be completed on smartphones, tablets, laptops, and desktops. Those who do not complete the survey by web will be contacted by an interviewer on the phone to complete the survey. Those who do not complete the survey during the phone outreach phase will then be contacted in person by an interviewer to complete the survey. In recognizing that sample members may have a preference to complete the survey in a certain mode, such as web, the web mode will continue to be available as an option for complete. For example, if an interviewer contacts a sample member by phone and the sample member would prefer to complete the survey on their own via the web, that option will still be available even though phone outreach is occurring.
Further, throughout data collection, the team will track response rates overall, by site, by mode, and by other factors which will allow the team to be agile in addressing nonresponse challenges. For example, if it is observed that text messaging is the best mode of communication for sample members, the team may adapt the protocol to include more text messages to reach more sample members to complete the survey. Or if it is observed that a specific site is lagging in response rates compared to the others, the team can take a deep dive into the data to identify the challenge and implement a solution, such as by examining the characteristics of respondents and nonrespondents using data available from the sample frame.
B6. Production of Estimates and Projections
Phase 1 and Phase 2 (Approved and In-Process; No changes from previously approved justification)
The data will not be used to generate population estimates, either for internal use or dissemination.
B7. Data Handling and Analysis
Data Handling
Phase 1 (Approved and In-Process; No changes from previously approved justification)
The five Fatherhood TIES grant recipient organizations use nFORM to collect electronic consent (Appendix #1) for Fatherhood TIES participants. Signed informed consents and nFORM data are accessed directly via the nFORM web portal, requiring a dual authentication for access. Responses to baseline survey (Instrument #1), staff reflection (Instrument #3) and father reflections (Instrument #4) are saved in MDRC’s FedRAMP-certified cloud-based environment. Access to the data is granted on a need-to-know basis and only the Data Manager and study team members with a need-to-know have access to the data. The Data Security Plan includes information on MDRC’s file logging system, data management, and file tracking. For instance, the Data Security Plan describes MDRC’s standard practices to access folders and files on a “need-to-know” basis. Also, data files with personally identifiable information (PII) are stored in folders within a FedRAMP/NIST 800-53-compliant environment. Only the Data Manager and a small number of trained programming staff have access to them. Thereafter, most data processing for the project will be performed on analysis files that have been stripped of direct identifiers.
Phase 2 (Current Request)
As submitted and approved in Phase 1, the project team will store data files including PII in the MDRC’s FedRAMP-certified cloud-based environment, Social Policy Research and Operations Unified Technology (SPROUT). For photo voice, participants will submit their own photos directly to MDRC through a secure electronic link using Qualtrics. For audio journaling, participants will submit their own audio recording data to a secure ZoomGov 1-800 line set up by MDRC. Audio recordings, photographs, and transcripts will be stored in dedicated folders within the FedRAMP/NIST 800-53-compliant environment. Access to the audio recordings and transcriptions will be granted on an “need-to-know” basis and only a small number of trained programming staff will have access to them. The data librarian will be the only MDRC staff in charge of logging the audio files and transcripts and transfer them to the dedicated folders within the FedRAMP/NIST 800-53-compliant environment.
The follow-up survey data will be collected using Abt’s Data Collection Platform (DCP) as well as the Data Collection and Analytic Computing Environment (DC-ACE), built and maintained on Amazon Web Services (AWS). The DC-ACE environment is highly secure and meets Federal Information Security Modernization Act (FISMA) Moderate security and compliance standards. The follow-up survey response data will be shared with MDRC either using Cerberus, or Box, the MDRC FedRAMP-compliant data-sharing system.
Data Analysis
Phase 1 (Approved and In-Process; Updates from previously approved justification)
Both nFORM data and baseline survey data are designed primarily for quantitative analysis. The baseline data (Instrument #1) will be used to monitor research group assignment (that is, program or control group assignment). Baseline data will also be used for the impact analysis in conjunction with the nine-month follow-up survey (Instrument #11) data to control for baseline characteristics and baseline equivalence. The team will see to what extent scores improved after program involvement. The nFORM program participation data will be used to monitor program participation and track sample members for the follow-up data collection.
Information from the program information and management tool (Instrument #2) and reflections from staff and fathers (Instruments #3 and #4) will be analyzed periodically throughout the study period to help inform the study team’s understanding of the implementation of the Fatherhood TIES interventions.
The study will be registered prior to the start of follow-up survey data collection with the American Economic Association.
Phase 2 (Current Request)
The nine-month follow-up survey is designed primarily for quantitative analysis as noted above. The data collected will be used to create outcome measures and run impact analyses. Exploratory analyses as well as confirmatory analyses will be performed to assess the validity and reliability of the scale measures collected for the study.
Periodic analysis of the qualitative instruments (Instruments #5 through 10) during the study period will help inform the team’s understanding of the implementation of the Fatherhood TIES interventions. This analysis will be particularly helpful in assessing the quality and fidelity of the interventions, including to identify changes over time.
Data Use
Phase 1 (Approved and In-Process; No changes from previously approved justification)
The information gathered via the program information and management tool and reflections from staff and fathers (Instruments #2, #3, and #4) will enable the study team to determine how well the interventions were implemented and whether there were any implementation problems. This will inform the study team’s technical assistance to the programs to strengthen their interventions.
Findings from Phase 1 and 2 will also be incorporated into documents or presentations that are made public, such as through conference presentations, website, or social media. These dissemination products may include infographics, short briefs or reports, or how-to guides and will describe how to properly interpret, analyze, and evaluate findings from the data collection, including limitations regarding generalizability and use of the study as a basis for policy.
This study will add to the body of knowledge in the fatherhood field, revealing evidence on whether program elements designed to produce positive outcomes for fathers do so.
Phase 2 (Current Request)
The implementation study analysis serves multiple functions: (1) to describe how each core component was integrated within existing program services and what it took to implement them throughout the study period; (2) to compare and contrast services provided to fathers who were randomly assigned to each core component test group with those assigned to the business-as-usual group; and (3) to draw on the experiences and perspectives of fathers and program staff to identify lessons from the field on core component implementation best practices and challenges.
The impact study has two overarching goals: 1) to estimate the impact of individualized supports (considering broadly) on fathers’ parenting, co-parenting/healthy relationships and overall well-being outcomes; and 2) to explore how different types of components (focused on program content, systems navigation, and engagement supports) improve a subset of target outcomes.
Findings from the current information collection request will be incorporated into documents or presentations that are made public, such as through conference presentations, website, or social media. These dissemination products may include infographics, short briefs or reports, or how-to guides and will describe how to properly interpret, analyze, and evaluate findings from the data collection, including limitations regarding generalizability and use of the study as a basis for policy.
B8.
Contact Persons
Dina Israel, MDRC
Fatherhood TIES Project Director, Principal Investigator
(212) 340 – 8606
Michelle Manno, MDRC
Fatherhood TIES Deputy Project Director, Co-Principal Investigator
(212) 340 – 8873
Charles Michalopoulos, MDRC
Fatherhood TIES Co-Principal Investigator
(510) 844 – 2235
charles.michalopoulos@mdrc.org
Attachments
Phase 1 (Approved and In-Process; No changes from previously approved justification)
Instrument #1 – TIES Baseline Survey
Instrument #2– TIES Table Template
Instrument #3 – TIES Reflection from Staff
Instrument #4 – TIES Reflection from Fathers
Appendix #1 – TIES Consent Materials for Fathers
Appendix #2 – IRB Approval
Phase 2 (Current Request)
Instrument #6 Staff Interview (including consent)
Instrument #7 Co-Parent Interview (including consent)
Instrument #8 Father Focus Group (including consent)
Instrument #9 Photo Voice Prompts, Consent and Release
Instrument #10 Audio Journaling Prompts, Consent, Release
Instrument #11 Nine-month Follow-Up Survey
Appendix #4 Photo Voice Training Slides
Appendix #5 Audio Journaling Training Slides
Appendix #6 Father Focus Groups Training Slides
Appendix #7 Photo Voice Training Guide
Appendix #8 Audio Journaling Training Guide
Appendix #9 Father Focus Groups Training Guide
Appendix #10 IRB Submission 3 Approval Letter
Appendix #11 IRB Submission 4 Approval Letter
1 Pending OMB approval.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Hjelm, Rebecca (ACF) |
| File Modified | 0000-00-00 |
| File Created | 2024-07-30 |