March 2008
Evaluation of the ACF Responsible Fatherhood, Marriage and Family Strengthening Grants for Incarcerated and Reentering Fathers and Their Partners
OMB
Supporting Statement
Office of Management and Budget Clearance
Package Supporting Statement and Data Collection Instruments
Prepared for
Office of the Assistant Secretary for Planning & Evaluation (ASPE)
United States Department of Health & Human Services (HHS)
Prepared by
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
RTI Project Number 0210412.000.013
Supporting Statement
Evaluation of the Administration for Children and Families Responsible Fatherhood, Marriage and Family Strengthening Grants for Incarcerated and Reentering Fathers and Their Partners
Submitted by:
U.S. Department of Health and Human Services
Office of the Assistant Secretary for Planning and Evaluation
Contact Person:
Linda Mellgren
(202) 690-6806
linda.mellgren@hhs.gov
Section Page
Background 1
A. Justification 1
A1. Circumstances Making the Collection of Information Necessary 1
A2. Purpose and Use of the Information Collection 2
A3. Use of Improved Information Technology and Burden Reduction 3
A4. Efforts to Identify Duplication and Use of Similar Information 4
A5. Impact on Small Businesses or Other Small Entities 4
A6. Consequences of Collecting the Information Less Frequently 5
A7. Special Circumstances Relating to the Guidelines of 5 CFR §1320.5 5
A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 5
A.8.1 Public Comment 5
A.8.2 Consultation with Experts 5
A9. Explanation of Any Payment or Gift to Respondents 6
A10. Assurance of Confidentiality Provided to Respondents 7
A11. Justification for Sensitive Questions 9
A12. Estimates of Annualized Burden Hours and Costs Error! Bookmark not defined.
A13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers 12
A14. Annualized Costs to the Federal Government 12
A15. Explanation for Program Changes or Adjustments 12
A16. Plans for Tabulation and Publication and Project Time Schedule 12
A.16.1 Plans for Tabulation 12
A.16.2 Plans for Publication 13
A.16.3 Project Timeline 14
A17. Reason(s) Display of OMB Expiration is Inappropriate 15
A18. Exceptions to Certification for Paperwork Reduction Act Submissions 15
B. Collection of Information Employing Statistical Methods 15
B1. Respondent Universe and Sampling Methods 15
B.1.1 Site Selection 15
B.1.2 Site-Specific Study Designs 18
B.1.3 Selection of Respondents 18
B.1.4 Power Analysis 20
B2. Procedures for the Collection of Information 21
B3. Methods to Maximize Response Rates and Deal with Nonresponse 25
B4. Tests of Procedures or Methods to be Undertaken 27
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 27
References 29
Number Page
1 Annual Burden Estimates Error! Bookmark not defined.
2 Annualized Costs to the Federal Government 12
3 Site Selection Criteria 17
4 Proposed Site-Specific Study Approaches and Projected Sample Sizes 19
5 Power Estimates from Monte Carlo Simulations 21
6 Site-Specific Implications of the Timing of Program Delivery and Participants’ Incarceration Terms on the Interview Schedule 22
Appendix A: Male Baseline Survey Instrument
Appendix B: Female Baseline Survey Instrument
Appendix C: Male Follow-up Survey Instrument
Appendix D. Female Follow-up Survey Instrument
Appendix E: Informed Consent Forms
Appendix F: Statute Authorizing Information Collection
Appendix G: Institutional Review Board Approval
Appendix H: Office of Human Research Protections Certification
Appendix I: Individuals and Organizations Consulted
Appendix J: Pilot Test Feedback Form
Appendix K: Overview of MFS-IP Programs
The Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the United States Department of Health and Human Services (HHS) is undertaking a study of Responsible Fatherhood, Marriage and Family Strengthening Grants for Incarcerated and Reentering Fathers and Their Partners, and is requesting clearance for six forms:
Male Baseline Survey Instrument (Appendix A)
Female Baseline Survey Instrument (Appendix B)
Male Follow-up Survey Instrument (Appendix C)
Female Follow-up Survey Instrument (Appendix D)
Male Informed Consent Form (Appendix E)
Female Informed Consent Form (Appendix E)
The purpose of the study is to evaluate grants designed to enhance partner and parenting relationships among incarcerated and reentering fathers, their partners and children. These grants are administered by the HHS Administration for Children and Families (ACF) Office of Family Assistance (OFA) under the authority of the Deficit Reduction Act of 2005 (DRA) (P.L. 109-171). The DRA amended Title IV, Section 403(a)(2) of the Social Security Act (42 U.S.C. 603(a)(2)) to authorize competitive grants for states; territories; Indian tribes; tribal organizations; and public and non-profit community entities, including faith-based organizations, to develop and implement projects that support any of the three authorized activity areas: Healthy Marriage, Responsible Parenting, and Economic Stability. ASPE has contracted with RTI International (RTI), and its subcontractors, to conduct the study.
The Marriage and Family Strengthening Grants for Incarcerated Fathers and Their Partners (MFS-IP) is one of the components of a broad policy strategy to support healthy marriage. The project will determine whether well-designed interventions can help couples in which at least one partner is incarcerated fulfill their aspirations for a strong relationship, and thus enhance the well-being of the entire family unit. In addition to supporting family wellbeing, research has shown that healthy relationships and family support may also be related to desistance from crime and more successful reentry into the community (Laub & Sampson, 2003; Visher & Travis, 2003; Sampson, Laub & Wimer, 2006).
The majority of men released from prison expect to live with their families, and most depend on their families for economic assistance, social support, and employment connections (Visher, La Vigne, & Travis, 2004). However, incarceration places a great deal of stress on family and intimate relationships. Couples are unable to engage in the day-to-day interactions and experiences that are necessary to sustain marital and other intimate relationships (Hairston, 2001). The risk of divorce is also very high among men going to prison; they attain the 50% divorce rate experienced by the general population in about one-third of the time. Separation, lack of intimacy, shame, anger, social stigma, and little understanding of the criminal justice system contribute to a high rate of marital dissolution (Hairston, 2001). Results from the Fragile Families and Child Wellbeing Study support these findings. Ever-incarcerated men were 43% less likely to marry and 42% less likely to cohabit compared with men who have never been incarcerated. Additionally, ever-incarcerated men were more likely to have separated from the mother of their child than never-incarcerated men (Western, 2004). These findings highlight the need for programs that teach important relationship-building skills and promote healthy marriages.
Incarceration affects the entire family, not only the partnered couple. Because of the record-breaking number of adults in prison, more children than ever are touched by the criminal justice system. Research reveals that there are nearly 3 million children in the U.S. with incarcerated parents (CCIP, 2001). Often, neither the children’s custodial or imprisoned parents are prepared to address the unique needs of these children (Hairston, 1995; Martin, 2001). Some children do not know their parent is incarcerated because relatives or incarcerated parents do not want the children to know, while some parents do not want their children to visit them in prison because such contact would be too emotionally painful (Hairston, 2001). Thus, many fathers in prison never see their children. Barriers such as lack of play space, long waits, body frisks and intrusive searches, and crowded and uncomfortable visiting rooms limit the quality and quantity of time incarcerated adults can spend with their children (Travis, McBride, & Solomon, 2005). Moreover, children face adjustment issues upon the parent’s release. The smooth transition of the reentering parent into the family unit is very important. In the context of relationships disrupted by incarceration, and potentially weak relationship skills, interparental conflict seems likely.
In order to provide effective institutional support for incarcerated and reentering fathers and their families, it is crucial to identify successful strategies for improving partner and parenting relationship outcomes in this population. The proposed information collection described in this statement will evaluate the programs currently funded by ACF in this priority area, so that information on the relative effectiveness of these programs or program components may be used to guide the design of future initiatives in this area. It is expected that such initiatives can play a key role in reducing the burden of incarceration on the families of incarcerated and reentering men.
Section 7103 of the DRA of 2005 provides that the Secretary of HHS may use the funds authorized under the amendment to Section 403 (A)(2) of title IV-A of the Social Security Act for the purpose of conducting and supporting research and demonstration projects. The legal authority for ASPE’s conduct of this study may be found in Section 301 of the Public Health Service Act (42 §USC 241) (see Appendix F).
The MFS-IP evaluation will accomplish several objectives. An implementation study of all 13 grantees will (1) describe the variety of programs and their corresponding grantee organizations on a number of dimensions including program history and context, type of grantee organization, target population, intervention strategies, and program design and (2) describe implementation fidelity and identify alterations from the original design, challenges, facilitators, and lessons learned. The implementation study will entail the documentation of start-up activities and ongoing implementation of the MFS-IP initiative through a combination of site visits, informal telephone interviews, and review of existing program MIS data. The implementation study will be customized for each site, based on unique site characteristics and local staff roles and responsibilities.
The impact study, which will be conducted in five of the sites, will (1) determine the impact of these diverse programs on outcomes such as marital stability, positive family interactions, family financial well-being, recidivism, and community connectedness and (2) identify the mediation mechanisms (or primary pathways) through which these programs achieve success. The primary source of data for the impact evaluation is a longitudinal study of approximately 2,000 couples (in which the male member of the couple is incarcerated at the beginning of the study) across the five sites. Half of the couples will be receiving marriage strengthening services through the MFS-IP grants, and half will be receiving either an alternate treatment or the standard set of services available in the correctional institutions. Both members of the couple (a total of 4,000 respondents) will be interviewed at baseline, nine months post baseline, and 18 months post baseline. Clearance is being requested for all four instruments that will be used for the study: male baseline survey, female baseline survey, male follow-up survey, and female follow-up survey. The 9- and 18-month follow-up instruments are identical in content. All study materials have been approved by the Institutional Review Board (IRB) at RTI (Appendix G) and certification has been received by the Office of Human Research Protection (Appendix H).
The data collection undertaken in this study will add to the healthy marriage-related literature by helping to determine what types of marriage programs work best for those involved in the criminal justice system; what does not work; and what effects these programs may have on fostering healthy, stable marriages and other intimate relationships, positive family interactions, and community connectedness. A better understanding of intervention research in the reentering population has great potential to generate insights that might result in more successful reentry of incarcerated individuals into the community and disrupt the intergenerational cycle that disproportionately reduces the life chances of low-income youth. Without the proposed data collection, HHS will not have evidence-based findings that can used to develop or refine effective program strategies serving this population, nor be able to assess whether these programs are effective and efficient.
The study will require collaboration with each of the program sites to streamline the collection of information required for the evaluation. The MFS-IP surveys will be administered via computer-assisted personal interviewing (CAPI) to 200 intervention and 200 control couples at each of the five MFS-IP impact sites. The use of CAPI will ease interview administration, ensure consistent administration, and maximize data quality through the provision of built-in editing. Anonymity and confidentially will be easier to protect using a computerized instrument compared with using a paper instrument. The data stored on the laptops will be encrypted and password protected. Moreover, the need for subsequent data entry will be eliminated, which reduces the potential for human error.
A web-based control system will be designed to track survey data. This system will allow RTI staff to monitor and record information regarding eligibility and participation rates, case and event information, and receipt of incentives. The advantages of using a web-based system for the survey data collection include the ability to access information from various locations, increased security as a result of login and identification requirements, and flexibility in terms of processing and reporting on the data.
Few studies to date have examined the effectiveness of interventions to improve partner and parenting relationships among incarcerated and reentering fathers and their partners. Several studies employing nonexperimental pre- and posttest designs have suggested the possible effectiveness of prison-based education programs to improve parenting and partner relationships (Hairston & Locket, 1987; Carlson & Cervera, 1991; Bayse et al., 1991; Accordino et al., 1998). For example, Bayse, Allgood and Van Wyk’s (1991) single group pre- and posttest evaluation found improved perceived family functioning and decreased narcissism among incarcerated men who participated in a family life education program. Accordino and Guerney (1998) found that incarcerated men and their partners who participated in a Relationship Enhancement curriculum reported positive perceptions of the program based on a one-time post-intervention evaluation.
Document and literature reviews were conducted, which confirmed that no scientifically rigorous evaluations of marriage and family-strengthening interventions with incarcerated fathers and their partners have been or are currently being conducted. A recent HHS report, Research and Practice Symposium on Marriage and Incarceration (Bauer, 2007) notes that very little is known about the effectiveness of marriage and family-strengthening programs with men being released from prison. Published reviews of the marriage strengthening literature concur with this finding. Larson (2004) described substantial gaps in the marriage education literature generally, including a lack of knowledge regarding the effectiveness of marriage strengthening with racial and ethnic minorities and working-class couples, as well as a dearth of experimental or quasi-experimental evaluations of marriage strengthening programs and a lack of understanding about which components of marriage strengthening programs produce the most change in relationship quality and stability. Bowling et al. (2005) found that the impact of cultural factors on marriage education has not been adequately evaluated. Markman et al. (2005) noted that studies of the effectiveness of marriage education programs with several types of couples were lacking, including those with “(a) very low income and high economic stress, (b) lower education levels, (c) ambiguous commitment between partners, including about a future and exclusivity, (d) a child out of wedlock, and (e) a male partner with serious problems, such as substance abuse, domestic violence, poor job history/opportunities, or a criminal history” (p. 127). In sum, there is little available information about marriage and family-strengthening efforts for incarcerated individuals and their partners.
Attendance at professional conferences confirmed that there are no other systematic research efforts underway on the effects of marriage and family strengthening activities for incarcerated and reentering men and their families. Conference attended include: Research and Practice Symposium on Marriage and Incarceration (Washington, DC, 2006), the National Institute of Justice Conference (Washington, DC, 2006, 2007), the American Society of Criminology Conference (Atlanta, GA, 2007; Los Angeles, CA, 2006), the International Family Violence and Child Victimization Conference (Portsmouth, NH, 2006, 2007), the African American Healthy Marriage Initiative “Connecting Marriage Research to Practice” Conference (Chapel Hill, NC, 2007), and the 2006 meeting of the American Correctional Health Services Association (Research Triangle Park, NC).
No small businesses or other small entities will be involved in the data collection.
This request is for baseline data collection and for follow-up data collection at nine months and eighteen months. Failure to collect this information would have serious consequences for the study, because the study aims to assess the effects of MFS-IP participation over time. It is essential to determine whether any impact achieved by the program extended for a reasonable period of time. Other current and well-regarded evaluations in both the corrections/reentry field and marriage/family field employ multiple follow-up periods and time points occurring well beyond program participation, in order to rigorously evaluate the impact of programming on outcomes. For example, the National Institute of Justice funded Multi-Site Evaluation of the Serious and Violent Offender Reentry Initiative (SVORI), involves in-person data collection among study participants at approximately one month prior to release and then again at three, nine, and 15 months after release, and the collection of administrative data at 24 months after release. In the marriage and family field, the evaluations of the Community Healthy Marriage Initiative, Building Strong Families, and Supporting Healthy Marriage all involve data collection for a three year follow-up period. In order to ensure the credibility of the results generated by the evaluation and provide scientifically-grounded guidance to the field on what type of marriage and family strengthening programming is most effective, longitudinal information collection at three time points (baseline and approximately nine and 18 months post-baseline) is necessary. By including multiple data collection periods in the impact study, the evaluation will be able to draw definitive conclusions about the sustainability of any positive effects achieved by the MFS-IP programs.
This request is for a one-time, serial information collection.
There are no special circumstances involved with this data collection.
A 60-day notice to solicit public comments was published in the Federal Register on December 3, 2007 (Volume 72, Number 231, page 67940). No comments were received during this period.
Consultations regarding the evaluation approach were held with eight scholars in the areas of incarceration and reentry, marriage and family strengthening, and evaluation design: Christy Visher, PhD, at the Urban Institute; Jeff Smith, PhD, at the University of Michigan; Creasie Finney Hairston, PhD, at the University of Illinois at Chicago; Randal Day, PhD, at Brigham Young University; Felix Elwert, PhD, at the University of Wisconsin, Madison; David Cordray, PhD, at Vanderbilt University; John Laub, PhD, at the University of Maryland; and William Fals-Stewart, PhD, at the University of Rochester (see Appendix I). A study design panel was held, at which experts provided feedback on the following aspects of the study approach:
Site-specific study design, including strategies for measuring the counterfactual at each site
Sample, enrollment, power and potential for pooling
Strategies for handling partner instability over time
Selection of a focal child for detailed parent report
Analytic approach
Based on input from the experts who were consulted, ASPE 1) confirmed the proposed study designs at each site, 2) determined that the original male and female members of each couple would be followed throughout the study period regardless of changes in relationship status, and 3) determined the criteria to be used to select a focal child.
Consultations regarding instrumentation were held with seven scholars in the areas of incarceration and reentry and marriage and family strengthening: Christy Visher, PhD, at the Urban Institute; Kristin Moore, PhD, at Child Trends; Jacinta Bronte-Tinkew, PhD, at Child Trends; Creasie Finney Hairston, PhD, at the University of Illinois at Chicago; Randal Day, PhD, at Brigham Young University; Howard Markman, PhD, at the University of Denver; and John Laub, PhD, at the University of Maryland (see Appendix I). Based on their input, new measures of mental health, affect, learning ability, family history, relationship quality, parental approval of the relationship, communication skills, and child well-being were added; reference periods were simplified; and the instruments were restructured chronologically to aid in respondent recall.
Respondents will be paid for completing each of three surveys: baseline and nine and 18 months post-baseline. The compensation schedule is as follows:
Male interviews (all waves): $35
Female interviews (all waves): $40
Bonus for calling the 1-800 number upon receipt of the lead letter: $5 (only available for non-incarcerated respondents)
Bonus for completing all three interviews: $25
The cash incentive for female respondents has been set at $40, $5 higher than for male respondents. Given the different obstacles to participation faced by male and female respondents (for example, child care and other obligations of community-based respondents), and the fact that males receive a more intensive intervention than females, provision of a higher incentive for female respondents is important in order to ensure comparable response rates for both members of study couples.
Negotiations with each state Department of Correction will be held regarding allowable compensation for respondents who are incarcerated at the time that they complete a survey. Where possible, survey incentive payments will be deposited in respondents’ inmate accounts. For sites at which this is not possible, permission will be sought for approval to provide the payment to a partner or family member on behalf of the inmate completing the survey. Respondents living in the community, including female partners and released male partners, will be paid in cash. Payment will be rendered to all respondents who make a good faith attempt to complete the interview, even if they do not finish the interview or refuse a significant number of questions.
Offering cash incentives to respondents will provide significant advantages to the study, including increased response rates at baseline and follow-up; reduced potential bias; and reduced data collection costs. The use of incentives for the MFS-IP survey is supported by the final report of the Symposium on Providing Incentives to Survey Respondents (1992) that was sponsored jointly by OMB and the Council of Professional Associations on Federal Statistics. The report recommended that OMB “seriously consider the use of incentives” for surveys that:
targeted difficult respondent populations, including surveys that have “small subpopulations of interest”;
surveys where response is affected by relatives who serve as gatekeepers to respondent access;
surveys for which participation may incur out-of-pocket costs, such as babysitting, to the respondent; and
surveys that are part of longitudinal panels.
The MFS-IP survey meets all of these guidelines: incarcerated individuals and their partners are a difficult population; the incarcerated individual serves as a gatekeeper for access to the partner; the partners are mostly custodial mothers who may incur out-of-pocket costs for care of children; and longitudinal data collection is planned.
The provision of a cash incentive for non-incarcerated respondents will make it convenient for a larger number of low-income sample members to make time for the interview during the defined data collection period, as opposed to using the time to take care of chores or generate income. This is confirmed by past research that showed that incentives are more effective for low-income households (Singer, 2002), although the evidence is mixed. Furthermore, incentives may counteract resistance among some respondents to participate in follow-up waves of the survey.
In addition to improving response rates, payment of incentives will reduce the cost of the survey to the government. The National Survey on Drug Use and Health (NSDUH), for example, attributed a net savings of $9 per interview and an increase in the weighted overall response rate from 68% to 71% for the 2002 survey when the use of a $30 incentive was approved by OMB. The cost of the MFS-IP study incorporates savings associated with reduced field travel costs and interviewer time required for repeated attempts at contact and refusal conversion due to the payment of incentive. As Warriner et al. (1996) noted, “At issue is not only the expense of the cash outlay for incentives but their effect on other fieldwork costs as well. The costs of follow-ups…means that some of the costs of incentives may be underwritten if an early response negates the need for further reminders.”
In summary, the project’s payment schedule represents appropriate compensation for respondent burden associated with completing the interview during the data collection period. This payment schedule will also ensure the collection of useful, cost-effective, and policy-relevant data for the MFS-IP evaluation.
Before the interview, each respondent will be asked to sign a consent form (Appendix E) that describes privacy protections at the beginning of the interview. As part of the informed consent process, each prospective study participant will receive information on the purpose of the survey, the expected length of time to complete the survey, the privacy protections of the information provided, and a name and telephone number at the contractor managing the survey for respondents to contact with any questions or concerns (separate numbers are provided for questions about the study and for rights of study participants).
Additionally, at the beginning of each interview all respondents will be told that: (1) their participation in the interview is voluntary; (2) there will be no penalties if they decide not to participate; (3) the information they provide will be combined with other participants’ answers and used only for the study; and (4) their names will never be used in reporting the results. The records produced in the study are not covered by the Confidential Information Protection and Statistical Efficiency Act (CIPSEA), so participants are not promised complete privacy of their records. The consent forms describe the exceptions to the procedures to protect privacy (“We may need to inform the appropriate authorities according to state and local law if you tell me that you are in immediate danger or that you intend to harm yourself or someone else or if I learn that a child is being abused or neglected or at risk for being abused or neglected.”) The consent forms also describe the use of a study identification number in place of a name.
Additionally, RTI, the contractor for this study, employs the following safeguards:
Project staff responsible for the data collection will be fully informed of policies and procedures regarding confidentiality of interview and records data. They will be trained to remind interview respondents of these procedures when they are administering these protocols.
All employees sign a confidentiality pledge that emphasizes the importance of confidentiality and describes their obligations. Releasing information that is confidential is considered a breach of trust and as such, employees may be subject to corrective action, up to and including termination of employment.
Identifying information for the purpose of locating respondents is maintained on separate forms (called contact sheets), which are linked to respondents by a sample identification number. Physical safeguarding and shipping procedures of these paper forms, and protocols for training interviewers in the use of these procedures, have also been developed. Access to the file linking sample identification numbers with the respondents’ identification and contact information is limited to a small number of individuals who have a need to know this information. These identification numbers will not be released outside RTI.
Access to sample selection data with personal identifying information is limited to those directly responsible for providing the sample and key staff who will be working with the sampling frame.
Access to computer files and hard-copy documents (such as contact sheets and locating packets) is strictly limited. Physical precautions include the use of PCs with password protection, secure areas, locked files and cabinets, and shredding of discarded materials.
The survey data collection procedures also minimize the risk of unauthorized disclosure of confidential data. The survey data will be collected via laptop. Respondents will be identified in the laptop’s case management system by the data collection case ID number and first and last name (in order to ensure that when field interviewers are pulling up the case identification number to conduct the interview, there is enough information to make sure they are pulling up the right case). Access to both the individual laptops and the case management system on the laptops will be password protected at each level. Each laptop hard drive will be encrypted using PointSec software. If a laptop is lost or stolen, data exposure will not occur unless the strong PointSec password is also compromised. Files with survey data will be identified only by the data collection case ID number (but will contain other identifying information gathered in the interview, such as date of birth and locating information).
For the central collection and organization of survey data, a master Blaise database will be maintained on the project share. Incoming data will be stored on the public file transfer protocol (FTP) server in an encrypted format and then moved, several times a day, inside a firewall to a private database on the project share for permanent storage. Access to this data is limited to in-house project staff and controlled by Windows authentication and a security group (LAN manager group). Data files and information stored on file servers will be protected and maintained as long as the project needs access. Following termination of the project, or following the end of the useful life of the information, files will be compressed, encrypted and archived to permanent media such as compact disk (CD) or digital video disk (DVD) for storage. CD and DVD archives will be maintained and stored in locked filing cabinets in the offices of project staff as long as needed, with destruction at the end of the study at the request of ASPE. All data will be treated as confidential. All personally identifiable information will be stripped from the file if a public-use data file is prepared.
Data with personal identifiers may be released to HHS agencies for purposes of data matching to obtain additional outcomes measures for MFS-IP study participants. If such data is released, HHS will ensure that a System of Records is in place covering individually-identified data collected for this study. This data will be transmitted securely, and will not be used for any purposes other than the aims of this study. At the conclusion of the research, these data are destroyed, and no identifying information will be made available in reports or other study publications.
These data collection procedures have been approved by an Institutional Review Board through RTI in Durham, NC (Appendix G), and by the HHS Office for Human Research Protections (Appendix H).
The aims of this study necessitate collecting data regarding mental health, substance abuse, and involvement with the criminal justice system, as well as domestic violence and other sensitive relationship experiences. This collection requires the inclusion of potentially sensitive questions regarding these topics in the male and female survey instruments at baseline and follow-up (Appendices A, B, C, and D). Such information is critical in order to assess the effectiveness of interventions designed to improve partner and parenting relationships and the mechanisms by which those effects, if any, occur. The following question areas, which are included in all four instruments, may be considered sensitive:
Intimate Partner Violence. The questions about domestic violence (see Domain L) and infidelity (see Domain U) are indicators of marriage quality and stability, which are key outcome variables for the evaluation.
Substance Use and Mental Health. The questions about substance use (see Domain N) and mental health (see Domain O) will provide contextual and explanatory information about MFS-IP impacts.
Criminal Activity and Criminal Justice Involvement. The questions about criminal activity and involvement with the criminal justice system (see Domains B, C, V and W) will also provide contextual and explanatory information about MFS-IP impacts.
Quality of Couple Relationship. Many of the relationship measures (see Domain U) were used in the baseline survey for the Evaluation of the Community Healthy Marriage Initiative (OMB #0970-0322).
SSN and Other Identifying Information. Respondent locating information, including social security numbers (SSNs) (see Locator Information) and birth dates (see Domain A), will be collected. Because baseline respondents will be followed up longitudinally, it is imperative that respondents can be found at a later date. SSNs will be necessary for searching electronic databases to learn the respondents’ locations, as well as to match to existing site-generated administrative data. Birth dates will confirm that the correct respondent has been located.
During the consent process, potential respondents will be informed of the topic areas of the interview and that some of the questions may make them feel uncomfortable or distressed. They will also be told that they can refuse to answer any of the questions by asking the interviewer to skip an item or items. The interviewers will be trained to identify signs of distress by listening to verbal and non-verbal indicators (e.g., shakiness in the voice, changes in volume, crying, etc.). If interviewers suspect that the respondent may be in distress or that the respondent’s safety will be compromised, interviewers will give the respondent the option to stop the interview or continue it at a later date. If the respondent’s level of distress persists or escalates, the interviewer will be prepared to follow the distressed respondent protocols developed for the study.
Several steps will be taken to minimize the risk of a breach of confidentiality during the interview. First, all interviews will be conducted in a private setting so that answers will not be overheard. For all interviews, privacy will be established (and documented via a series of questions built into the instrument) to ensure the confidentiality of the interview. The interviewers will be trained on what constitutes an acceptable setting for the interview and how to handle any disruptions or violations of the privacy that has been established. Community-based interviews may be conducted in the respondent’s home or another private setting such as a study room in a local library. An interview with one member of the couple will not be conducted with the other member of the couple present in the home. No interviews will be allowed in any criminal justice settings except jails or prisons. Whether interviewing in a community or facility setting, before beginning the interview, the instrument will prompt the interviewer to check if they are out of earshot of other people including people who may be in an adjacent room or outside. The interviewer must be seated in such a way that no on can walk behind him or her to view the computer screen and must be completely confident that the respondent’s answers cannot be overheard or ascertained by someone looking through a window.
Second, for all interviews conducted outside of correctional facilities, Audio Computer-Assisted Self-Interviewing (ACASI) will be used for the particularly sensitive portions of the interview including questions pertaining to domestic violence, substance use, and other illegal behaviors. Respondents will use headphones to hear a voice recording of the questions while reading along on the screen and will enter their own answers into the computer. This methodology has proven to be a highly successful means of gathering sensitive information.
Finally, the interviewer training will strongly emphasize the privacy and confidentiality aspects of the study and all project staff (including interviewers) will be required to sign a pledge of confidentiality.
Two hundred couples participating in MFS-IP programs at each of five sites will be surveyed over the course of 54 months. Members of a comparison population of 200 non-participating couples at each of five sites will also be surveyed over the same time period.
All male participants will be incarcerated at baseline. Therefore, we estimate their wages at $0.69 per hour, which is the midpoint of the wage range paid to inmates in federal prison industries (Economic Policy Institute, 2008). We assume that female study participants will be primarily low income and that their hourly pay rates will, on average, be the same across study intervals. Therefore, the wage estimate for female participants at baseline and follow-up is based on an annual income of $34,340, which is 200% of the 2008 federal poverty level for a three-person household; this translates to an hourly rate of $17.17. The wage estimate for male participants at follow-up assumes that participants will have been released from prison and earning, on average, the federal minimum wage of $5.85 per hour. Exhibit 1 reflects burden estimates for each instrument, annualized across the 54-month data collection period.
Exhibit 1. Annual Burden Estimates
Instrument |
Number of
Respondents |
Number
of |
Average Burden Hours per Response |
Total
Annualized |
Hourly Wage Rate |
Annualized Hourly Costs |
Baseline survey for incarcerated men |
444.4 |
1 |
1.5 |
666.7 |
$0.69 |
$460.02 |
Baseline survey for partners of incarcerated men |
444.4 |
1 |
1.5 |
666.7 |
$17.17 |
$11447.24 |
Follow-up survey for incarcerated or formerly incarcerated men |
444.4 |
2 |
1.5 |
1333.3 |
$5.85 |
$7780.00 |
Follow-up survey for partners of incarcerated or formerly incarcerated men |
444.4 |
2 |
1.5 |
1333.3 |
$17.17 |
$22892.76 |
TOTAL |
|
|
|
4000.0 |
|
$42580.02 |
There are no other costs to respondents.
The total cost to the federal government for the impact study is estimated to be $6,705,381.00 (Exhibit 2) over the entire 6-year study period. Of the total costs, $6,367,881.00 is for developing and administering the evaluation study, including survey design, data collection, analysis, reporting and necessary meetings. Approximately $337,500.00 is for federal oversight provided by ASPE. Costs below are annualized over six years, which includes the 4.5-year data collection period, as well as study activities (listed above) that take place immediately prior to and following the data collection.
Exhibit 2. Annualized Costs to the Federal Government
Costs Directly Related to Conducting Project |
$6,367,881.00 |
Project Oversight by Federal Government |
$337,500.00 |
Total Costs |
$6,705,381.00 |
Total Annualized Costs |
$1,117,563.50 |
This is a new submission.
The study will employ analytic approaches that combine data gathered from individuals and their partners in the longitudinal impact study with existing administrative data (e.g., child support, employment, and recidivism data) to determine program effectiveness. This integrated analysis will facilitate the understanding of what specific programmatic components or models appear to be more successful than others at affecting key outcomes. The study will present simple comparisons between treatment and control/comparison groups, adjust the comparisons using multivariate models, and use hierarchical models to address the multiple levels of interest, including the nesting of observations on one person within the couple, in each time period. Multiple measures of certain constructs will be collected, self-reported data will be compared with administrative data, reports from couple members will be compared, and the self-reported sequencing of behavior, time use and spending patterns will be explored. Comparisons between sites and with the control groups from other Healthy Marriage Evaluations may also be useful for understanding the context of any treatment effects that are found.
Analysis of the various data collected for treatment and control/comparison groups in each site will include a progression from simple to more sophisticated methods. The first step of the analysis will be to present tabular differences between groups. Second, the mean differences will be adjusted in a regression context. Third, individual fixed effects to isolate within individual changes over time will be added. Lastly, strategies to isolate treatment effects relative to changes in the control group over time will be used. These analyses will also be performed by subgroup if power allows. Hierarchical data methods and survival analyses illustrating differential time to re-incarceration and propensity score methods for adjusting on observables within these methods may also be of interest. The analytical approach will incorporate several strategies to manage possible selection bias (see B3, below).
The analysis may allow for pooling data in two separate ways. First, across all five sites pooling may be possible to test the average treatment effect related to participation in any marriage and family strengthening program, to get information on relative dose and intensity of programs in different sites, and for the assessment of the relative effectiveness of holistic programs that embed marriage education within a broader service menu, as opposed to those that focus only on marriage education services. The decision as to the feasibility of pooling to test for average treatment effect will be made after closer examination of variation in intervention design and target population may limit the cross-site data pooling.
Second, pooling may be possible across two of the impact grantees whose programs are similar in mode and content:
Osborne Association and Indiana Department of Correction, where couple-based services consist of an eight hour couples-based marriage education course delivered in a single weekend, and where post-release services are not provided; and
Shelby County Division of Correction, Minnesota Council on Crime and Justice, and New Jersey Department of Corrections, where couple-based services include intensive case management along with marriage education and other ancillary services, and where couples are provided with services in the community for at least six months after the male partner’s release.
A final decision regarding the usefulness of, and best approach to, pooling data from the sites will be made on the basis of a detailed analysis of implementation and program design features.
The evaluation approach and findings will be communicated to a diverse audience, including interested practitioners and researchers in the fields of criminal justice and marriage and family strengthening. Products to be produced and disseminated during the study include an implementation report and an impact study report. The impact study report will include detailed information on the data collection methods, survey results, and recommendations regarding the effectiveness of family and marriage strengthening programs for incarcerated parents. The report will be published on the ASPE website, and hard copies will be distributed at conferences and upon request.
Additionally, eight practice briefs and four special topics reports related to the evaluation will be produced. Practice briefs will provide research-based, yet accessible information geared at states and communities that are conducting or considering implementing initiatives that target healthy relationships among incarcerated or formerly incarcerated fathers. The first practice brief will present an overview of the national evaluation strategy and give a brief synopsis of grantee programs. The second practice brief will provide a more thorough description of programs being implemented by each of the 13 grantees and will propose a typology of program models based on this information. Selection of other topics will be based on identification of priority issues that can inform policy and practice. Approximately two briefs or special reports each year will be issued in order to maintain a consistent flow of information from the project.
Presentations will also be made at conferences and meetings. Presentations on the evaluation were made to the MFS-IP grantees at the initial grantee meeting (December 2006) and at the Training and Technical Assistance meeting (July 2007). Presentations will be made at subsequent yearly grantee meetings (expected May 2008, 2009, 2010). Other venues for presentations about the evaluation include the annual ACF Welfare Research Conference, the African American Healthy Marriage Research and Practice Conference, the Association for Public Policy and Management Conference, and the American Society of Criminology meeting.
This study will take place over six years, with enrollment in the longitudinal study to occur on a rolling basis for three years and data collection to continue over 54 months. The following schedule highlights key study milestones during that period.
Negotiate & Sign Memoranda of Understanding with Grantees January 4-
March 28, 2008
Program, Test and Finalize Computer Interviewing Module January 12-
June 27, 2008
Obtain Local IRB Approvals February 1-
April 25, 2008
Begin Administering Baseline Survey September 19, 2008
2 months after OMB approval
Begin Administering 9-Month Survey June 19, 2009
11 months after OMB approval
Begin Administering 18-Month Survey March 19, 2010
20 months after OMB approval
Submit Final Implementation Report April 30, 2011
Submit Final Impact Report December 6, 2013
This information collection effort does not ask to be exempted from displaying the OMB expiration date.
No exceptions to the certification statement are requested.
The target population for the survey is couples participating in the MFS-IP program. The couples are comprised of incarcerated men and their partners (who are not incarcerated). Questionnaires will be administered to the men and to their partners in separate interviews. All couples who enroll in couple-based services funded by the MFS-IP initiative at five selected program sites will be recruited to participate in the study. A cohort of comparison or control couples who are not participating in MFS-IP services will also be recruited at each of the five sites. Recruitment and baseline interview will take place during the period of incarceration. During the follow-up period some men may still be in prison, while others may have been released from incarceration. The service settings and target populations vary among the grantees, and the sites vary widely in terms of services delivered and service delivery approach (Exhibit 3). The broad set of program components planned among the set of grantees includes marriage education, marriage/family counseling, parenting education, enhanced visitation options, case management, education and employment services, support groups and cultural activities, financial literacy services, mentoring and coaching services, and domestic violence services. Brief program descriptions are included in Appendix K.
Data will be collected from program participants and control/comparison group members at a subset of five grantee sites: Shelby County Division of Correction (Memphis, TN), Minnesota Council on Crime and Justice (Minneapolis, MN), New Jersey Department of Corrections (Trenton, NJ), Osborne Association (Brooklyn, NY), and Indiana Department of Correction (Indianapolis, IN) (Exhibit 3). These five sites were selected based on six criteria.
Each program had to include a couple-based relationship focus because the impact evaluation must achieve its goal of determining whether couple-based family strengthening programming has a positive impact on relationship quality, child well-being, and other outcomes.
Program intensity was looked at because in order for a program to achieve any desirable outcomes, a reasonable level of program exposure/contact is necessary.
Program enrollment was a key factor, as it is in any evaluability assessment. Sufficient numbers of couples receiving the treatment (as well as a sufficient number of “untreated” couples to serve as the comparison/control group) are an important prerequisite for acceptable statistical power for detecting actual treatment effects.
Stage of implementation was a consideration because modifications to program design or delivery once the evaluation is underway are extremely undesirable. Therefore, it was necessary that programs have finalized plans for program delivery (and be ready to begin implementing their programs or have already begun implementation) in order to be selected for evaluation.
Study design considerations were key. Specifically, we assessed whether each program was willing to randomly assign eligible couples to receive MFS-IP programming or “treatment as usual,” and if not, whether other possibilities for the identification of a comparison group existed at the site.
Finally, the site’s willingness to participate and staff capacity for participation were essential considerations because the impact evaluation cannot be conducted successfully without the cooperation of the participating sites.
Exhibit 3. Site Selection Criteria
Impact Study Sites |
Target Population |
Program Components |
Rationale |
NJ DOC |
Targets men incarcerated at three state facilities who are either in a committed relationship with children or married, have six to nine months left to serve, have been identified as having an addiction problem, and are max-out offenders. Men may only participate if their partners agree to enroll. |
Participants will receive case management, including visitation coordination, discharge planning, and substance abuse counseling; a 12-week marriage education and parenting curriculum (Married and Loving It); a post-release substance abuse treatment program; and referrals to support groups and marriage counseling. |
|
MN CCJ |
Targets fathers admitted to St. Cloud state prison who are in a committed relationship, are from and returning to the Twin Cities area, and have a sentence of six months to three years. Men may only participate if their partners also enroll. |
Program components, all of which are provided to both members of the couple, include the following: case management, parenting classes, relationship classes, financial literacy training, and employment referrals, training, and placement. |
|
Shelby County DOC |
Targets men incarcerated in the Adult Offender Center of Shelby County prison who have six to 12 months left to serve and are in a committed relationship, and their partners. |
All program participants receive parenting education, employment skills training, moral reconation therapy, and case management. Most participants receive a marriage strengthening course (PAIRS), money management education, family group conferencing, GED/higher education classes. The PAIRS component is optional for those men whose partners cannot attend; in the first cohort, about half of the men were participating in PAIRS with their partners. Child-friendly visitation is also available to interested participants who are fathers. |
|
Osborne Association |
Targets incarcerated fathers at three New York DOCS facilities (Sing Sing, Fishkill, and Greenhaven) and their partners. |
Those who enroll in the parenting class receive 16 weeks of classroom-based fatherhood training. Healthy Relationships class participants receive five or six weekly sessions. Those who take part in the couples PREP program receive an eight-hour class delivered during a single weekend. |
|
IN DOC |
Targets men incarcerated at 13 state prisons who are participants in character/faith-based living units (PLUS) or general inmate population graduates of a parenting class, and their partners. |
Couples retreat participants will receive eight hours of marriage education (PREP) delivered jointly to both members of the couple over a single weekend.
|
|
Each of the selected sites has a strong couple-based relationship program, acceptable enrollment numbers, and strong possibilities for counterfactual strategies. Individually, they fulfill all key selection criteria and collectively they offer a diverse set of programs—one for which key questions can be answered regarding the effectiveness of couple-based relationship-strengthening services in general and specific program components that appear to be associated with greater effectiveness.
Within the five impact sites, negotiations are currently underway to finalize the most robust design for each site, particularly in terms of the counterfactual strategy. Several strategies have been discussed internally and externally, with external study design experts involved in the discussions. Proposed approaches for each site are detailed in Exhibit 4. It will be essential to understand and document the counterfactual in each site, since it is unlikely that comparison and control group members will avoid services altogether. Service receipt by control and comparison group members will be documented as part of the implementation evaluation (see B2, below), and through the inclusion of a battery of items regarding service receipt in the survey instrument (see Appendices A–D). The specific strategies will be different in each site but the general approach is to apply the basic selection criteria used to identify program participants to individuals who are not eligible for the program for reasons such as 1) being incarcerated in a facility in which the program is not offered, 2) being from a community other than the community targeted by the program, or 3) having a release date outside of the parameters established by the program.
Selection of subjects for the longitudinal interview component of the evaluation will be conducted with the assistance of each site participating in the impact evaluation. For “treatment” couples (i.e., those who have been officially enrolled in the MFS-IP programs), program staff will provide contact information to the study for the men and their partners. For “comparison” couples, program or agency research staff will assist in the identification of eligible subjects. If a surplus of eligible study members is identified for either the treatment or comparison group, we will either truncate the baseline enrollment period in the site or randomly sample from the eligible respondents.
Exhibit 4. Proposed Site-Specific Study Approaches and Projected Sample Sizes
Grantee |
Proposed Approach |
Projected Sample Size |
NJ DOC |
The program currently enrolls two cohorts per year, limited to inmates with projected release dates of six to nine months (at the time of recruitment) and who are eligible based on several additional characteristics. Random assignment is one possibility in this site, but such an approach would reduce the number of treatment couples from 270 over the three years to 135 each for the treatment and control group. Therefore, an alternative strategy being considered are inmates who meet all program eligibility criteria but who fall within the two annual recruitment cycles because their release dates are more than six to nine months at the first recruitment cycle and less than six months at the second recruitment cycle. This group is otherwise identical to the comparison group and the slight differences in overall duration of incarceration are not a large source of bias. |
200 treatment and 200 comparison couples over the three year enrollment period |
MN CCJ |
The program currently enrolls inmates from the seven county Twin Cities area into their program (with several other eligibility criterion applied). After eligible men are screened, partners are recruited and then the couple is enrolled. Random assignment at the individual level (i.e., prior to the recruitment of partners) is one possibility at this site (random assignment after the recruitment of partners is undesirable from the site’s perspective due to the ethical issues involved in building partner’s trust only to be not selected for participation), but such an approach would not yield equivalent treatment and comparison groups because some “treatment” group members would end up not actually enrolled in the program (because the partner may end up not consenting to enroll). Therefore, an alternative strategy being considered is having the program refine its geographical eligibility criteria such that inmates from three counties are eligible for the program and inmates from the remaining four counties are eligible for the evaluation study only. Recruitment procedures (and all other eligibility criteria) would be identical for the two groups. |
180 treatment and 180 comparison couples over the three year enrollment period |
IN DOC |
The program serves residents of a specialized housing unit and general population inmates who have completed a parenting class, across 13 state prisons. In order to be eligible for the PREP couples retreat (the component being evaluated), men must be in a committed relationship (with a partner who is willing to attend the retreat). Because the retreat is offered at each facility only twice a year, a possible comparison group strategy is men who want to participate in the retreat but who are released before the retreat can be offered. A large pool of inmates (across the 13 facilities) makes DOC data matching a possibility to identify inmates most comparable to the treatment group. Random assignment is unlikely due to the very small number of men at each facility who are currently enrolling in the retreats and the lack of enrollment caps on the retreats. |
220 treatment and 220 comparison couples over the three year enrollment period |
Osborne |
The PREP couples retreat (the component being evaluated) is available to men in a committed relationship (with a partner who is willing to attend the retreat) who have completed prerequisite parenting and healthy relationship classes. Because there is not a surplus of couples eligible for the retreat (and the enrollment targets for this site are low), random assignment is unlikely. Alternative design approaches include screening graduates of parenting classes at comparable NY DOC facilities not served by the grant and identifying men in committed relationships who would be interested in attending a couples retreat. |
200 treatment and 200 comparison couples over the three year enrollment period |
SC DOC |
The program serves inmates at the Adult Offender Center in a county prison. Because there is not a surplus of men eligible for the program (and the enrollment targets for this site are low), random assignment is unlikely. An alternative comparison group is inmates in the main facility of the prison who are participating in a fatherhood program offered only to inmates in that building. The program, funded under OFA’s Priority Area 2 initiative, includes child-friendly visitation, parenting education and some of the ancillary services that are offered to participants in Project REACH, which is funded under OFA’s Priority Area 5. Research question would focus on the added impact of a marriage intervention for couples. |
200 treatment and 200 comparison couples over the three year enrollment period |
Estimates of power can be difficult with many of the multilevel analyses proposed; in several instances, closed power formulae for the designs proposed have not been developed. However, power can always be estimated using Monte Carlo simulations, where multiple data sets, using bootstrap resampling, are generated using estimates of parameters provided by the user to inform the simulation. In turn, the multiple data sets are analyzed to determine power (Thomas & Krebs, 1997). To remain consistent, this “brute force” approach, which is described fully in Peladeau and Lacouture (1993), was used to estimate power for all of the proposed analyses. Because we have conducted related studies using some of the measures we plan to use here, effect size estimates are generally available. The power analyses presented here use pooled estimates of parameters and standard errors from the following studies (a) Behavioral Couples Therapy for Drug Abuse, (b) Abbreviated Couples Therapy for Drug Abuse, and (c) Group-Based BCT for Drug Abuse which are all NIDA-funded longitudinal trials. The following assumptions are made in this analysis:
There is a 30% missing rate at any given time point, with missing data addressed using Full Information Maximum Likelihood methods (and adjusting, of course, for increased standard error due to missing data)
The analysis uses a longitudinal design comparing couples therapy to an equally intensive Treatment-As-Usual (TAU)
The models assume Couple Dyadic Adjustment Scale (DAS) scores nested within time; the models are very similar if only husband or wife scores are used, so there is no gain with other methods
The model assumes normal data (DAS tends to look somewhat normal)
These multilevel models will be estimated in a covariance structure framework (versus traditional multilevel framework, as implemented in statistical programming packages such as HLM or MLwiN)
The analysis uses a .05 alpha.
Power estimates in Exhibit 5 are from Monte Carlo simulations using procedures described in Sartorra and Saris (1985) relating to the DAS and the Conflict Tactics Scale (CTS II). For the DAS and CTS (as well as most primary outcome measures on children’s adjustment, parenting behavior, substance use, employment, legal entanglements, and so forth), the effect sizes observed in Dr. Fals-Stewart’s trials are medium-sized using Cohen’s (1988) conventions. In these contexts, a medium effect size for the DAS translates roughly to an increase in scores of about 15 to 20 points more in the treatment group than the control group. In categorical terms, this means a move from very distressed to distressed, distressed to mildly distressed, mildly distressed to normal, normal to happy, or happy to very happy. For the CTS, it means about 20% more couples in the treatment group will not commit any acts of partner violence in the post-intervention period compared with controls. In both cases, this is benchmarked against an active TAU and these effect sizes are viewed as clinically meaningful. This effect size would be detected with 80% power in samples of 200 or more couples in treatment and control conditions.
Exhibit 5. Power Estimates from Monte Carlo Simulations
Sample Size (in terms of couples per condition; assumes 2 conditions): |
DAS |
CTS II1 |
50 |
0.62 |
0.4 |
100 |
0.76 |
0.56 |
150 |
0.86 |
0.69 |
200 |
0.94 |
0.8 |
250 |
0.98 |
0.87 |
300 |
0.99 |
0.92 |
The study is designed to assess the short- and long-term effects of participation in MFS-IP programming on key outcomes, including relationship quality and stability, intimate partner violence, parenting behaviors, child well-being, family income, and recidivism. Several relationship quality constructs will be measured, including relationship satisfaction (using the DAS-8), fidelity, marriage plans, positive couple interactions, supportiveness, shared decision making, and conflict. The relationship stability domain will include measures of marital history, marital status, presence of a romantic relationship, cohabitation, and commitment. Several variables posited as mediators between MFS-IP programming and the outcomes above will also be measured, including substance abuse, employment, parenting skills and attitudes toward marriage.
Interview data will be obtained from each member of participating couples. Baseline interview data will be collected before initiation of any MFS-IP services. Follow-up data will be collected at nine and 18 months post-baseline. The timing of the follow-up data collections from the point of enrollment is appropriate because the interventions vary widely in terms of their length, dose, and timing relative to the term of incarceration. The follow-up surveys (Appendices C-D) are structured so that all respondents receive some common sections, while some sections are administered only to respondents who are still incarcerated or only to respondents who have been released. Detail on the timing of each interview wave relative to program delivery and study participant incarceration term is provided in Exhibit 6.
Exhibit 6. Site-Specific Implications of the Timing of Program Delivery and Participants’ Incarceration Terms on the Interview Schedule
Program |
Baseline Interview |
9 Month Interview |
18 Month Interview |
The Osborne Association |
|||
Timing re: program delivery |
Immediately after enrollment |
Program likely completed for all |
At least 12 months after program completion |
Timing re: incarceration term |
Variable (enrollment not based on sentence characteristics and can take place any time during incarceration) |
Variable |
Variable |
IN DOC |
|||
Timing re: program delivery |
Immediately after enrollment |
Program likely completed for all |
At least 12 months after program completion |
Timing re: incarceration term |
Variable (“enrollment” into couples retreat takes place for most at the end of the 4th quarter [12 months] of PLUS participation) |
Variable |
Variable (but most likely coming up on release, given average sentence lengths and the fact that they have likely already served a year at program entry) |
SC DOC |
|||
Timing re: program delivery |
Immediately after enrollment |
Program still ongoing for all |
Program ends for all by this time period |
Timing re: incarceration term |
Six-12 months from release |
Within three months prior to three months after release |
Six-12 months after release |
NJ DOC |
|||
Timing re: program delivery |
Immediately after enrollment |
Program still ongoing for all |
Program ends for all at this time period |
Timing re: incarceration term |
Seven-ten months from release |
Within one month prior to release to two months after release |
Eight -11 months after release |
MN CCJ |
|||
Timing re: program delivery |
Immediately after enrollment |
Program still ongoing for all |
Program still ongoing for all |
Timing re: incarceration term |
Immediately after intake; one-three years from release |
All are still incarcerated |
From up to 18 months before release to six months after release |
The recruitment of study participants will occur on a rolling basis, and enrolling 200 treatment and 200 control couples in each of the five sites is expected to take three years (the end point of which coincides with the end of the sites’ grants). Therefore, several waves of data collection will overlap, and the overall data collection will take 54 months.
The study will utilize field supervisors and field interviewers from RTI’s National Interviewer File, and be supplemented with new hires as necessary to afford full coverage of a site. Field supervisors will attend a one-day in-person training session focused on project management responsibilities. Then all field supervisors and field interviewers will attend a five-day, in-person training session, covering procedures for contacting respondents, gaining cooperation, avoiding and converting refusals, administering the interview, and reporting. Training will involve a combination of lecture, demonstration, and hands-on skills practice. All field supervisors and field interviewers will be required to pass a certification exam upon the completion of training.
Interviewers (and their associated field supervisors) will receive an electronic contact summary report for all of their assigned baseline cases. Specific recruitment procedures differ for incarcerated and non-incarcerated respondents. For eligible respondents who are incarcerated, they will be brought individually to a private room in the correctional facility where the study and all respondent rights will be explained. It will have been relayed ahead of time to the facility contact (in the facility access negotiations carried out by site liaisons) that facility staff should not convey any details about our study to the potential participants who have been identified. When approaching potential candidates about meeting with the interviewer, facility staff will tell the inmates that a researcher would like to meet with them and talk about the possibility of doing an interview for a research study and that the researcher will tell them more about the study in the interview room. The field interviewer will go over the consent form and answer any questions that the respondent has about the study. Prison officials will be told that the length of the interview varies greatly (from ten minutes to two hours).
Eligible respondents who are not incarcerated (which includes most female respondents at baseline and follow-up, as well as some male respondents at follow-up) will be mailed a letter introducing or reminding them about the study. The letter will contain a toll-free number that the respondent may call to schedule the interview and increase his or her payment by $5. If the respondent does not call the interviewer within a week of the letter being sent, the interviewer will attempt to contact the respondent (via telephone or a home visit) to schedule the interview. Copies of the lead letters (a baseline version, which will only be used for partners, and a follow-up version, which will be used for all non-incarcerated respondents) and the Q&A brochures (which will not accompany the partner lead letters but rather be used by the field interviewer when meeting with the partner in person to discuss whether she is interested in participating in the study) are attached, in addition to the “Sorry I Missed You” cards and appointment cards that will be used in the field for setting up interviews.
Interviews will be conducted in a private setting. For incarcerated respondents, (all male participants at baseline), the interview will be conducted in a private room at the correctional facility, and only the respondent and the interviewer will be present. For partners, the interview will be conducted at the respondent’s home or other private location. Men who are eligible for study participation will be escorted individually to a private room in the correctional facility where they reside. The interviewer will then describe the study to potential participants.
For all interviews, after confirming that the correct respondent is present, participants will be handed the consent form and will read along as the interviewer reads the consent form text directly from the laptop. The consent form will display the interview consent text and a signature line for participants to indicate their consent to be interviewed (and for the interviewer to sign). The consent form also has additional text and a separate signature line for the respondents to indicate their consent for having random segments of the interview audio recorded for quality control purposes (i.e., Computer-Assisted Recorded Interview [CARI]). Participants will sign one copy of the consent form for the project files and retain an unsigned copy for themselves.
The interviewer will administer the interview in a prescribed and uniform manner. The interviews will be conducted using Computer-Assisted Personal Interviewing (CAPI). For particularly sensitive sections of community-based interviews (which will include most respondents to the baseline and follow-up female surveys, and some respondents to the follow-up male surveys), ACASI will be used. Specifically, the ACASI section includes the questions on criminal history/behavior, criminal/drug involvement of the people with whom the respondent resides, substance use, relationship fidelity, and intimate partner violence. For the non-ACASI sections, the field interviewer (FI) will read the questions from the screen and enter the respondent’s answers into the laptop. If at any time, the privacy of the interview setting is compromised, the interviewer will pause the interview until privacy can be reestablished, rescheduling as necessary. Each interview will last approximately one-and-a-half hours and will cover the following topics: basic demographic information, attitudes, programs and services, family structure, relationship quality, parenting, physical and mental health, substance use, criminal behavior, employment and income, expectations for release, and future contact information. The content of all instruments (male and female, baseline and follow-up) is similar. However, the time periods about which the questions are asked differ, and there are separate skip and fill patterns depending on whether the respondent is incarcerated.
At the conclusion of community-based interviews, respondents will be given their payment and asked to initial a receipt. At the conclusion of facility-based interviews, FIs will follow the compensation procedures allowed by the facility (e.g., no compensation, a money order deposited in the inmate’s account, or payment to a community designee).
During the course of the study, FIs may observe respondent distress or child abuse or neglect. Critical incident protocols have been developed for the study (with separate versions for facility- and community-based interviews). These protocols specify steps the interviewer and other project staff charged with decision-making should follow. The interviewers will be trained extensively on these protocols.
Some portions of each interview will be recorded by the laptop using CARI technology. The purpose of CARI is to detect interviewer falsification. CARI files for 5% of all interviewers’ cases will be reviewed by the project quality control manager. The respondent’s permission to use CARI will be requested during the informed consent process. The respondent may still participate in the interview even if he or she declines CARI. If the respondent agrees to CARI, as a necessary condition of detecting interviewer falsification, neither the respondent nor the interviewer will be aware of when the computer is making the recordings. At least three 30-second portions of the interview will be recorded as well as several responses to “other specify” questions. CARI will not be used during the portion of the interview that asks the respondent for future contact information. CARI will also not be used during any portion of the interview that asks particularly sensitive questions.
The field supervisors for 5% of community-based interviews will also conduct standard telephone verification these interviews.
In any longitudinal evaluation, much depends on retaining the comparability of the treatment and control groups over time as well as on understanding the sample composition over time. There are two reasons why the treatment groups may have higher response rates than the control groups:
The issues of marriage, family, and child well-being may be more salient in the treatment group since some of the interventions may enhance this awareness.
Individuals who have benefited from the interventions who might otherwise have refused to respond to the survey may decide to participate to reciprocate for perceived benefits; likewise those who have not benefited as much from the interventions will tend not to cooperate with the survey request.
There maybe other reasons why the response rates may differ, but these two reasons represent the heart of the issue. Both reasons have the potential for biasing the estimates of the treatment effects. Increased salience might lead certain groups that typically have low response rates (such as low education and some minority groups) to respond at a higher rate if they are in the treatment group relative to the control group. Additionally a better response rate from those who benefited more from the interventions could overstated the benefits and the estimated treatment effect may be biased upward.
The impact study will implement two strategies to address these concerns. First, a data collection methodology designed to minimize the non-response bias will be used, particularly as it affects the comparison of estimates between control/comparison and treatment groups. Here it is important to note that having the same response rates in both treatment and control/comparison groups does not guarantee that bias in the treatment effect is minimized. This is because non-response bias is the product of two components: the non-response rate and the difference in the characteristics of respondents and non-respondents. Thus, the differential non-response bias in treatment versus control comparisons may still not be zero if the compositions of the non-responding populations are different with respect to the characteristics of interest. To guard against this eventuality, a commitment will be made to successfully contacting and screening sample members and achieving the highest possible response rates. Such methods include:
In-Person Interviewing. It is expected that in general the prospective participants will be in a lower socioeconomic class than average, and that some of them may not have regular telephone access. When surveying a hard-to-reach population of this kind, experience has shown that an interviewer-administered mode yields higher response rates than self-administered modes. In-person interviews using CAPI will be the most efficient means of efficiently gathering interview data.
Respondent Convenience and Multiple Attempts. Interviews will be scheduling interviews at the respondent’s convenience. There will be multiple attempts to reach nonrespondents, including leaving “Sorry I Missed You” cards at empty households at the time of a scheduled visit. As the study progresses and the best times to reach respondents and make contact attempts is ascertained, the interviewer scheduling will be adjusted accordingly.
Customized Lead Letters. Customized lead letters will be sent in advance of fielding to promote respondent cooperation. The lead letter will explain the study objectives, explain that the survey is voluntary, and assure confidentiality. Moreover, the letter will provide several means for respondents to contact interviewers, including a toll-free telephone number and email address.
Financial Incentive. A cash incentive will be offered to each respondent who makes a good faith effort to complete the survey. It is anticipated that this will increase perceived benefit so that respondents will make time for the interview. The incentive payment will also help emphasize the importance of participating in the study.
Comprehensive Interviewer Training. Interviewing staff will participate in a multi-day, comprehensive training. Interviewers will be trained on the study purpose and procedures, interview administration, and the protection of human subjects. Past literature has shown that interviewer effect can be a source for potential survey bias. Therefore, a thorough understanding of the study and the instrument, and upholding standard protocols and ethical commitment will reduce bias and in turn help interviewers gain respondent trust.
Refusal Aversion and Conversion. Part of the interviewer training will address in detail specific techniques to avert and convert a refusal from a respondent. Respondents who initially refuse to participate will be assigned to interviewers who have a proven record of turning refusals into completed interviews. Reasons for refusals and barriers to participation will be continually evaluated in light of the experience gained in the data collection process.
Regular Debriefings with Data Collection Staff. The project management staff will regularly meet with data collection staff to discuss issues related to data collection operations. Methods to enhance response rates will be a standard agenda item at these meetings.
To the extent possible, response rates by pre-identified demographic and other variables during data collection will be continually monitored to ensure that both groups at a site have not only the same response rates but also the same patterns of nonresponse across the demographic groups.
If a differential rate or bias is apparent in the estimates, statistical adjustments will be used to further minimize the risks of bias. Response propensity models will allow the weighting of the data so that the demographic composition of both groups in a site are statistically equivalent on those variables that are most highly correlated with and most critical to analysis. Another approach involves sample selection models, both static and dynamic. Here, the goal is to take account of the factors influencing selection into the sample in a first stage equation and then include a selection variable in an outcome equation.
Difference between groups on observable variables will be adjusted statistically using propensity matching and other methods. Unobservable differences between treatment and control groups are more concerning, but our screening collection will offer a partial solution in that it captures equal motivation to receive services between participants. Other impact estimation methods will be used that attempt to address selection bias by using alternative comparisons and counterfactuals. Possible approaches will include comparing the outcomes of the entire eligible population in the presence of the program with the outcomes of that population without the program, comparing different cohorts of participants because participation can be lagged by as much as three years, and matching treatment and comparison groups on observable characteristics.
In developing the instruments for this study, a pilot test of the male and female baseline survey instruments was conducted. The purpose of the pilot was to test the overall flow and length of the survey, to elicit information on how respondents are forming and reporting responses, and to identify any questions that may be difficult for respondents to answer. The male instrument was piloted with a convenience sample of nine incarcerated men who were married or in a committed relationship and had minor children. The female instrument was piloted with the committed partners of those men who agreed to do the interview and agreed to provide their partners’ contact information. Male interviews were conducted in the correctional facility where respondents were incarcerated, and female interviews were conducted in respondents’ homes. Both versions were piloted using a paper-and-pencil version of the instrument. Respondents were paid $25 each for the interviews. The instruments were revised based on information obtained through this process, with the goal of improving the quality of data collected and minimizing burden on respondents. Reference periods were simplified for some constructs to provide greater ease of recall for those items that pilot study participants found difficult to answer. The incentive process was also tested, and payment for female respondents was increased to $40.
Most items included in the data collection instrument have been used successfully in previous studies with similar populations, including the Multi-Site Evaluation of the Serious and Violent Offender Reentry Initiative (SVORI), the Evaluation of the Community Healthy Marriage Initiative (CHMI), and the longitudinal Returning Home study of reentering prisoners. Investigators who were involved in each of these projects have provided guidance and feedback on the survey instrument, and their experiences collecting survey data involving similar populations and outcomes of interest has been very helpful.
a) Individuals who have participated in designing the data collection:
ASPE staff
Linda Mellgren Linda.Mellgren@hhs.gov
Jennifer Burnszynski Jennifer.Burnszynski@hhs.gov
Nicole Gardner-Neblett, PhD Nicole.Gardner-Neblett@hhs.gov
Diana Merelman Diana.Merelman@hhs.gov
RTI International Staff
Anupa Bir, PhD abir@rti.org
Christine Lindquist, PhD lindquist@rti.org
Tasseli McKay tmckay@rti.org
b) Individuals who will participate in the collection of data (all from RTI International):
Kristine Fahrney fahrney@rti.org
Field supervisors and interviewers will be hired for the data collection.
c) Individuals who will participate in data analysis:
ASPE Staff
Linda Mellgren Linda.Mellgren@hhs.gov
Jennifer Burnszynski Jennifer.Burnszynski@hhs.gov
Diana Merelman Diana.Merelman@hhs.gov
RTI International Staff
Anupa Bir, PhD abir@rti.org
Christine Lindquist, PhD lindquist@rti.org
Pamela Joshi, PhD joshi@rti.org
Mindy Herman-Stahl, PhD mindy@rti.org
Kristine Fahrney fahrney@rti.org
Tasseli McKay tmckay@rti.org
Accordino, M.P., and B. Guerney. 1998. “An Evaluation of the Relationship Enhancement Program with Prisoners and Their Wives.” International Journal of Offender Therapy and Comparative Criminology 42(1):5-15.
Bauer, B., J. Hart, A. Hopewell, and N. Tein. 2007. “Research and Practice Symposium on Marriage and Incarceration: A Meeting Summary.” Prepared by Health Systems Research Inc. for the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, January 2007. <http://aspe.hhs.gov/hsp/07/marr-incar/index.htm>.
Bayse, D.J., S.M. Allgood, and P.H. Van Wyk. 1991. “Family Life Education: An Effective Tool for Prisoner Rehabilitation.” Family Relations: Journal of Applied Family & Child Studies 40(3):254-257.
Bowling, T.K., C.M. Hill, and M. Jencius. 2005. “An Overview of Marriage Enrichment.” The Family Journal: Counseling and Therapy for Couples and Families 13(1):87-94.
Carlson, B.S., and N. Cervera. 1991. “Inmates and Their Families.” Criminal Justice and Behavior 18(3):318-339.
Center for Children of Incarcerated Parents (CCIP). 2001. CCIP Data Sheet 3a: How Many Children of Incarcerated Parents Are There? Eagle Rock, CA: Center for Children of Incarcerated Parents. <http://www.e-ccip.org/publication.html>. As obtained on June 29, 2006.
Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
Economic Policy Institute. (2008). Economic snapshots. Snapshot for August 21, 2002: Bringing the jobs back home to prisons. Retrieved March 7, 2008, from http://www.epi.org/content.cfm/webfeatures_snapshots_archive_08212002
Hairston, C.F. 1995. “Fathers in Prison.” In Children of Incarcerated Parents, D. Johnston and K. Gables, eds., pp. 31-40. Lexington, MA: Lexington Books.
Hairston, C.F. 2001. “Prisoners and Families: Parenting Issues during Incarceration.” Chicago, IL: Jane Addams College of Social Work, University of Illinois, Chicago. <http://aspe.hhs.gov/search/hsp/prison2home02/Hairston.htm>.
Hairston, C.F., and P.C. Locket. 1987. “Parents in Prison: New Directions for Social Services.” Social Work 32(2):162-164.
Harrison, K. 1997. “Parental Training for Incarcerated Fathers: Effects on Attitudes, Self-Esteem, and Children’s Self Perceptions.” Journal of Social Psychology 137:588-593.
Larson, J.H. 2004. “Innovations in Marriage Education: Introduction and Challenges.” Family Relations 53:421-424.
Laub, J.H., and R.J. Sampson. 2003. Shared Beginnings, Divergent Lives: Delinquent Boys to Age 70. Cambridge, MA: Harvard University Press.
Markman, H.J., G. Kline, J.G. Rea, S. Simms-Piper, and S.M. Stanley. 2005. “A Sampling of Theoretical, Methodological and Policy Issues in Marriage Education: Implications for Family Psychology.” In Family Psychology: The Art of the Science, W.M. Pinsof and J.L. Lebow, pp. 115-137. New York: Oxford University Press.
Martin, J.S. 2001. Inside Looking Out: Jailed Fathers’ Perceptions about Separation from Their Children. New York: LFB Scholarly Publishing LLC.
Peladeau, N., and Y. Lacouture. 1993. “SIMSTAT: Bootstrap Computer Simulation and Statistical Program for IBM Personal Computers.” Behavior Research Methods, Instruments & Computers 25(33): 410-413.
Sampson, R.J., J.H. Laub, and C. Wimer. 2006. “Does Marriage Reduce Crime? A Counterfactual Approach to Within-Individual Causal Effects?” Criminology 44(3):465-508.
Sartorra, A., and W.E. Saris. 1985. “Power of the Likelihood Ratio Test in Covariance Structure Analysis.” Psychometrika 50:83-90.
Singer, E. 2002. “The use of Incentives to Reduce Non-Response in Household Surveys.” In Survey Nonresponse, R. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, eds. New York: Wiley.
Snyder, Z.K., T.A. Carlo, and M.M. Coats Mullins. 2001. “Parenting from Prison: An Examination of a Children’s Visitation Program at a Women’s Correctional Facility.” Marriage & Family Review 32(3-4):33-61.
Thomas, L., and C.A. Krebs. 1997. “A Review of Statistical Power Software.” Bulletin of the Ecological Society of America 78:126-139.
Travis, J., E.C. McBride, and A.L. Solomon. 2005. The Hidden Costs of Incarceration and Reentry. Washington, DC: The Urban Institute, Justice Policy Center. <http://www.urban.org/publications/310882.html>. As obtained on June 29, 2006.
Visher, C., N.G. La Vigne, and J. Travis. 2004. Returning Home: Understanding the Challenges of Prisoner Reentry, Maryland Pilot Study: Findings from Baltimore. Washington, DC: The Urban Institute, Justice Policy Center. <http://www.urban.org/publications/ 410974.html>. As obtained on June 29, 2006.
Visher, C.A., and J. Travis. 2003. “Transitions from Prison to the Community: Understanding Individual Pathways.” Annual Review of Sociology 29:89-113.
Warriner, K., H. Goyder, H. Gjertsen, P. Hohner, and K. McSpurren. 1996. “Charities, No; Lotteries, No; Cash, Yes: Main Effects and Interactions in a Canadian Incentives Experiment.” Public Opinion Quarterly 60:542-562.
Western, B., L. Lopoo, and S. McLanahan. 2004. “Incarceration and the Bonds Between Parents in Fragile Families.” In Imprisoning America: The Social Effects of Mass Incarceration, M. Patillo, D. Weiman, and B. Western, eds., pp. 21-45. New York: Russell Sage Foundation.
1 The power estimates for the CTS II (a measure of partner violence) require the additional assumption of a negative binomial model for the very highly skewed characteristics that usually mark violence data.
| File Type | application/msword |
| File Title | Division of Violence Prevention |
| Author | tmckay |
| Last Modified By | DHHS |
| File Modified | 2008-04-08 |
| File Created | 2008-04-08 |