supporting statementOklahoma
supporting statementOklahoma.doc
Oklahoma Marriage Initiative Process Evaluation
OMB: 0990-0309
FP
N
Oklahoma Marriage Initiative Process Evaluation
OMB Supporting Statement for Data Collection
Draft
December 8, 2006
Shawn E. Marsh
Heather H. Zaveri
Debra Strong
M. Robin Dion
Submitted to:
U.S. Department of Health and Human Services Office of the Assistant Secretary for Planning and Evaluation Rm. 404E, 200 Independence Ave., SW Washington, DC 20201
|
Submitted by:
Mathematica Policy Research, Inc. 600 Maryland Ave., SW, Suite 550 Washington, DC 20024-2512 Telephone: (202) 484-9220 Facsimile: (202) 863-1763
Project Director: Robin Dion |
|
|
CONTENTS
Section Page
A. JUSTIFICATION 1
1. NEED AND LEGAL BASIS FOR the Data Collection 1
2. How, by Whom, and for What Purpose Information will be Used 4
3. Use of Automated Electronic, Mechanical, and Other Technological Collection Techniques 5
4. Avoiding Duplication of Effort 5
5. Sensitivity to Burden of Small Entities 5
6. Consequences to Federal Program or Policy Activities if the Collection is not Conducted or is Conducted Less Frequently than Proposed 5
7. Special Circumstances 7
8. Federal Register Announcement and Consultation 7
9. Payments and Gifts to Respondents 8
10. Confidentiality of the Data 8
11. Additional Justification for Sensitive Questions 9
12. Estimates of the Hour Burden of the Collection of Information 9
13. Estimate of Total Annual Cost Burden to Respondents or Record Keepers 12
14. Estimate of Annualized Costs to the Federal Government 12
15. Reasons for Program Changes or Adjustments 12
CONTENTS (continued)
Section Page
16. Plans for Tabulations and Publication and Schedule of Project 12
17. Reasons for not Displaying Expiration Date for OMB Approval 14
18. Exception to the Certification Statement 14
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 15
1. Respondent Universe and Sampling 15
2. Procedures for the Collection of Information 19
3. Methods to Maximize the Response Rate and to Deal with Nonresponse 22
4. Tests of Procedures and Methods to be Undertaken 23
5. Individuals Consulted on Statistical Aspects of the Design 23
APPENDIX A: AUTHORIZING LEGISLATION
APPENDIX B GUIDE FOR SEMI-STRUCTURED INTERVIEWS WITH LEAD STAFF AT PUBLIC AGENCIES
APPENDIX C: GUIDES FOR GROUP INTERVIEWS WITH WORKSHOP LEADERS
APPENDIX D: GUIDE FOR GROUP INTERVIEWS WITH COMMUNITY VOLUNTEER WORKSHOP LEADERS
APPENDIX E: GUIDES FOR FOCUS GROUPS WITH PROGRAM PARTICIPANTS
APPENDIX F: GUIDE FOR FOCUS GROUPS WITH COMMUNITY WORKSHOP PARTICIPANTS
CONTENTS (continued)
Section Page
APPENDIX G: PARTICIPANT INFORMATION FORMS
APPENDIX H: FEDERAL REGISTER NOTICE
A. JUSTIFICATION
The Office of the Assistant Secretary for Planning and Evaluation (ASPE) at the U.S. Department of Health and Human Services is requesting clearance for data collection instruments to be used in conducting a process evaluation of the Oklahoma Marriage Initiative (OMI). The OMI is the country’s largest and most longstanding effort of its kind, preceding the national Healthy Marriage Initiative declared by President Bush in 2001. Even before the national effort began, other states have looked to Oklahoma’s pioneering effort for guidance on how to build similar statewide initiatives. Unfortunately, little information has been available due to the lack of an in-depth process evaluation that could identify implementation lessons and report on the OMI experience. As this document describes, the data collection for which ASPE is requesting clearance is designed to address this specific gap in our knowledge base and provide information that could prove valuable to other states and localities planning similar initiatives.
1. NEED AND LEGAL BASIS FOR the Data Collection
The data collection proposed in this application is authorized under Section 301 of the Public Health Service Act (42 USC 241) (see Attachment A). The data are needed because there is currently a dearth of information about how to design and implement community-wide marriage initiatives—information that could prove valuable to the hundreds of similar efforts that are beginning as a result of new federal funding for marriage initiatives. The OMI is the largest and most longstanding initiative of its kind and yet has received almost no research attention. Information gained in this process evaluation of Oklahoma’s initiative is expected to provide lessons that may influence the quality and evidence base of new initiatives in other states and communities.
Marriage initiatives are a relatively new phenomenon in the world of public policy, which have begun to proliferate as a result of recently passed congressional legislation allocating $150 million annually to states and organizations to implement marriage and responsible fatherhood programs under the federal Healthy Marriage Initiative. Interest in understanding the OMI model and what has been learned during its implementation is expected to intensify because although several evaluations of future initiatives are planned, few large-scale programs currently exist that could inform questions related to development, design, and implementation. In addition, the OMI is so far the only initiative to attempt implementation of services throughout an entire state. From its outset, the OMI took a unique and innovative approach to promoting and strengthening marriage by using millions of dollars from its Temporary Assistance for Needy Families (TANF) block grant to implement a vast service delivery system involving the use of hundreds of volunteers and engaging the participation of multiple public agencies as well as private organizations (Myrick and Ooms 2002).
This process evaluation will include a special focus on one aspect of the OMI: the delivery of marriage education services to incarcerated and re-entering individuals and their partners. Marital and parent-child relationships are at high risk of disruption during parental incarceration and re-entry, and the number of prisoners who are parents has been growing in the U.S. About 650,000 people are released from prison each year, while 7 million cycle through jails. The Bush Administration and Department of Health and Human Services (HHS) are interested in improving the rehabilitation and re-entry success of these individuals as they transition back to their families and communities. Initiatives have so far focused on job training, mentoring, and other services such as the President’s Re-Entry Initiative, the Serious and Violent Offenders Re-entry Initiative, and the Mentoring Children of Prisoners. The Responsible Fatherhood, Marriage and Family Strengthening Grants for Incarcerated Fathers and their Partners will provide funding for a range of states and organizations to provide marriage education and related services to the prisoner and ex-offender population. At this time, the OMI is the only statewide marriage program that includes a focus on the needs of prisoners and their partners. Very little is known about how prison-based programming may need to differ from other marriage education programs being developed for low-income populations. Examining how Oklahoma developed, revised, and refined its approaches and strategies to serve this population will inform the next stage of marriage education services for this population.
Including the focus on services for prisoners, the overall purpose of the OMI process evaluation is to document and analyze the context from which the initiative developed; its overall goals and objectives; and its evolution, organizational structure, partnerships, and service delivery system. ASPE has contracted with Mathematica Policy Research, Inc. (MPR) to conduct the data collection and analyze the results. Throughout this process study, MPR will focus on three purposes that are of primary interest to ASPE:
To document and chronicle the OMI from conception to the development and implementation of its program strategy
To document and analyze the OMI’s program strategy and design, and understand as well as possible the consequences of the approaches taken
To build upon this understanding to identify lessons and implications for both the continuation of the OMI and the development of initiatives in other states
With these objectives in mind, MPR will explore the context in which the OMI developed, its overarching strategy for implementation within multiple sectors, the role of research and information in developing an implementation strategy, and OMI goals for changing the systems and culture within which it operates. Three general types of data collection are required for this primarily qualitative study: (1) Semi-structured interviews with lead program staff at public agencies; (2) group interviews with workshop instructors; and (3) focus groups with workshop participants.
The objectives of the OMI process study will be pursued by focusing on eight main research questions:
How did the Oklahoma context shape the philosophy, strategy, and organization at the start of the OMI and in ongoing course adjustments?
How did the OMI build vision and broad support for focusing on marriage
What is the overarching OMI philosophy, and how is it reflected in overall strategy, implementation decisions and methods, and adjustments over time?
What are the reasons for and implications of focusing on marriage education as a vehicle for change, and of choosing a single curriculum as the foundation for the OMI?
What have been the major accomplishments, benefits, and challenges of implementing the OMI through both volunteer-based and institutional avenues?
What part has been played by research and information in the OMI’s implementation?
What goals for changing systems and culture does the OMI address? What responses has the OMI elicited in the form of workshop leadership and workshop participation?
2. How, by Whom, and for What Purpose Information will be Used
ASPE anticipates that the information obtained through the proposed data collection will be used by MPR to produce a series of “practice briefs” targeted to practitioners who are implementing similar initiatives in other jurisdictions, and a final comprehensive report to document the OMI and suggest lessons from Oklahoma’s experience to inform state and federal policy and program efforts and OMI leaders themselves.
3. Use of Automated Electronic, Mechanical, and Other Technological Collection Techniques
Since the data collection for this study is primarily qualitative, no automated data collection techniques will be used. Electronic, mechanical and other technological techniques are not applicable to our data collection activities.
4. Avoiding Duplication of Effort
There is no similar, prior or ongoing data collection being conducted that duplicates the efforts of the proposed data collection. The interview protocols and focus group guides do not ask for any information that can be obtained through abstraction of existing records.
5. Sensitivity to Burden of Small Entities
While some of the entities that sent volunteers for training would be considered small businesses, we are not asking these entities to participate in the data collection, and will not create any burden for them.
6. Consequences to Federal Program or Policy Activities if the Collection is not Conducted or is Conducted Less Frequently than Proposed
The Oklahoma Marriage Initiative is the country’s largest and most longstanding statewide marriage initiative. The proposed data collection is a one-time effort to interview key individuals involved with the design and implementation of the OMI; the option for less frequent data collection is no data collection. If the proposed data are not collected, the loss of potential knowledge may have adverse consequences for current national policy efforts to implement marriage education programs in general, and programs for the incarcerated and re-entering population in particular. With millions of dollars being invested in marriage education programming under the President’s Healthy Marriage Initiative and HHS services for incarcerated and re-entering populations, the government must take every opportunity to learn from existing initiatives. The OMI is the only statewide initiative that provides services on large scale in multiple sectors of society; other states seeking to replicate this model would not have available the necessary information to guide their efforts. They would not, for example, understand what implementation strategies were successful, what challenges Oklahoma encountered, and what pitfalls they should look out for and avoid. Failing to collect the proposed data would result in a loss of information that could potentially be very valuable to current federal program and policy activities.
If we do not collect the information through semi-structured interviews and focus groups as described, it will be very difficult for federal policymakers, practitioners, and grantees to learn from Oklahoma’s experiences in this emerging policy arena, and their efforts to do so will be unduly burdensome to the OMI. To obtain this information on their own, interested parties would first have to obtain and cull through disparate materials relevant to the OMI from numerous reports, publications, and other sources. Second, they would have to spend their own resources and time obtaining and analyzing data or reports from Oklahoma administrative sources. Third, they would need to identify, contact, and interview individuals in the various state and local, public and private sector entities involved in the OMI, to learn about why certain choices were made, and how Oklahoma’s environment and experience might play out or need to be adapted for their states. Each interested state or entity would have to replicate these efforts on its own. Since these activities would be resource-intensive for each inquiring state, and burdensome for Oklahoma, states would be less able to take advantage of Oklahoma’s experience to frame their own exploration of this new policy field. This would result in longer and more costly learning curves for each state, which will burden providers, participants, and taxpayers or other program funders. Thus, although there are no legal obstacles to reduce the burden, it is more efficient for MPR to collect and analyze data pertaining to the OMI, and for MPR and ASPE to work together to disseminate the information.
If the data cannot be collected, we also will be unable to address the government's key evaluation questions. Instead, we could only review and summarize existing data and information already presented by PSI or the state of Oklahoma, which does not fully address the evaluation questions posed by ASPE. In addition, not collecting the new information would prevent us from fulfilling the main purposes of the study, which is to provide outside, objective information to the state of Oklahoma and to others interested in implementing marriage initiatives.
7. Special Circumstances
There are no special circumstances.
8. Federal Register Announcement and Consultation
A request for comment on the proposed data collection activity and instruments was published in the Federal Register on August 28, 2006 (Vol. 71, No. 166, p. 50921). A copy of the notice is provided in Appendix H.
a. Comments
There were no comments received from the Federal Register Announcement.
b. Consultation with Experts
During preparation of the study design, we engaged the professional counsel of several people. These consultations included Dr. Sameena Salvucci and Dr. Barbara Devaney on sample design, and Alan Hershey on evaluation design.
c. Unresolved Issues
None.
9. Payments and Gifts to Respondents
We plan to offer compensation to the participants in the focus groups conducted with workshop participants. We will offer $35 to participants in focus groups to reimburse respondents for their time and expenses. Participants will be giving up substantial personal time and may incur expenses related to childcare and transportation. For these reasons, it is important to offer incentives commensurate with the time and expense associated with participation. In some cases, incarcerated respondents may not be permitted to receive such a gift. An alternative arrangement will be made in such cases (e.g., a gift to the prisoner’s family).
10. Confidentiality of the Data
The Privacy Act will not apply to the data collected in this study, because the federal government will not collect and maintain the data as a system of record. Instead, confidentiality of the data will be assured and maintained following standard MPR procedures, which are consistent with the provisions of the Privacy Act. Each respondent will be asked to sign a consent form that describes confidentiality protections at the beginning of the interview or focus group. Additionally, at the beginning of each focus group or interview all respondents will be told that (1) their participation in the interview or focus group is voluntary; (2) the information they provide will be held in strict confidence and used only for the study; (3) their name will never be used in reporting the results; and (4) their answers will not be released in any manner that would enable someone to identify them, unless they give written consent or the law requires.
Project staff responsible for the data collection will be fully informed of MPR’s policies and procedures regarding confidentiality of interview and records data. They will be trained to remind interview respondents of these procedures when they are administering these protocols. MPR routinely employs the following safeguards to ensure confidentiality:
All employees at MPR sign a confidentiality pledge that emphasizes the importance of confidentiality and describes their obligations. MPR views releasing information that is confidential or that is proprietary to the company as a breach of trust and as such, employees may be subject to corrective action, up to, and including, termination of employment.
Access to sample selection data with personal identifying information is limited to those directly responsible for providing the sample. At the conclusion of the research, these data are destroyed.
To the extent that locating is necessary for recruiting, identifying information is maintained on separate forms (called contact sheets), which are linked to respondents by a sample identification number. Access to the file linking sample identification numbers with the respondents’ identification and contact information is limited to a small number of individuals who have a need to know this information.
Access to computer files and hard-copy documents (such as contact sheets and locating packets) is strictly limited. Physical precautions include the use of PC’s with password protection, secure areas, locked files and cabinets, and shredding of discarded materials.
Application is being made for Institutional Review Board approval of the data collection procedures through Public/Private Ventures in Philadelphia, PA.
11. Additional Justification for Sensitive Questions
There are no sensitive questions in the protocols being used in this study.
12. Estimates of the Hour Burden of the Collection of Information
Table 3 presents the number of respondents, the number of responses per respondent, the average burden hours per respondent, the total annual burden hours for the data collection, the average hourly wage rate by respondent, and the total respondent costs. Data for the process study will be collected through semi-structured interviews with staff at lead agencies, group interviews with workshop leaders and focus groups with program participants. Data will also be collected by program observation when possible and by reviewing existing reports and information. These latter activities will not place additional burden on the sites. Most of the information will be collected during site visits.
TABLE 3
ANNUAL BURDEN ESTIMATES
Instrument |
Maximum |
Number |
Average Burden Hours |
Total |
Hourly Wage Rate* |
Total Respondent Costs |
Protocol for Lead Staff at Public Agencies |
44 |
1 |
1.50 |
66 |
$18.57 |
$1,226 |
Guide for Group Interviews with WS Leaders at Public Agencies |
84 |
1 |
1.50 |
126 |
$14.97 |
$1,886 |
Guide for Group Interviews with Community WS Leaders |
24 |
1 |
1.50 |
36 |
$14.80 |
$533 |
Focus Group Guide for WS Participants at Public Agencies |
84 |
1 |
1.50 |
126 |
$7.98 |
$1,005 |
Focus Group Guide for Community WS Participants |
24 |
1 |
1.50 |
36 |
$14.80 |
$533 |
Estimated Total Annual Burden Hours and Total Respondent Costs |
390 |
|
$5,183 |
|||
*Note: Hourly wage rates are from the “Oklahoma Quarterly Census of Employment and Wages, 2004,” which was released in May 2006 (available on the internet at http://www.oesc.state.ok.us/lmi/publications/Qcew/QcewBook04.pdf). The hourly wage rate for lead staff at public agencies represents the statewide average hourly wage rate in the public administration sector; for workshop leaders at public agencies the rate represents the statewide average hourly wage rate for health care and social assistance workers; for community workshop leaders and participants the rate represents the statewide average hourly wage for all workers across all employer sizes; for workshop participants at public agencies the hourly wage rate was calculated from the 2006 Federal Poverty Guidelines for a family of 3.
The time burden estimate for the semi-structured interviews with lead staff at the public agencies is 66 hours. These interviews will occur during site visits, and each may involve multiple respondents, ranging from two to four people, in order to limit the amount of time spent at the site. Thus, the maximum annual burden for the semi-structured interviews with lead staff at public agencies will be 11 (interviews) x 4 (participants per interview) x 1.5 hours = 66 hours. The burden for the group interviews with workshop leaders at public agencies is 126 hours and in the community the burden is 36 hours for a total burden of 162 hours. The attendance goal for each group interview with workshop leaders is eight to 12 people. Seven group interviews will be held with workshop leaders at public agencies, and two will be held with workshop leaders in the community, totaling 9 group interviews, with a total of up to 108 respondents. Each group is expected to last approximately 90 minutes. Thus, the maximum annual burden for the group interviews with workshop leaders will be 9 (interviews) x 12 (participants per interview) x 1.5 hours = 162 hours.
The attendance goal for the focus groups with workshop participants is the same as for the group interviews with workshop leaders. For each focus group with participants at public agencies and in the community, we estimate that eight to 12 people will attend. We anticipate that each focus group will last approximately one and one-half hours and that we will conduct seven focus groups with public agency participants and two focus groups with community-volunteer participants. Thus, the annual burden for the focus groups will be 9 (focus groups) x 12 (participants per focus group) x 1.5 hours = 162 hours.
13. Estimate of Total Annual Cost Burden to Respondents or Record Keepers
Telephone calls to recruit focus group participants will be placed at the expense of the evaluation contractor, and respondents who wish to call the research team will be provided with a toll‑free number billed to the contractor. Respondents will not incur any out-of-pocket costs.
14. Estimate of Annualized Costs to the Federal Government
All costs for conducting the process study of the Oklahoma Marriage Initiative will be borne by the evaluation contract between DHHS and Mathematica Policy Research, Inc. (Contract No. HHSP233-02-0086), plus the personnel costs of federal employees involved in oversight. The estimate of average annual cost is as follows:
$ 154,225 Contract Staff salaries (for developing data collection protocols, planning and conducting data collection, analyzing data, and reporting results)
$ 12,205 Contract Staff Travel
$ 47,769 Contract Other
$ 16,000 Federal oversight of data collection, analysis, and reporting (15 percent of FTE)
______________________________
$ 230,199 Total cost for 12 months
15. Reasons for Program Changes or Adjustments
This is a new project which incorporates new research.
16. Plans for Tabulations and Publication and Schedule of Project
a. Publications
Findings from this study will be reported in two ways. First, lessons from the OMI experience will be disseminated through a series of practitioner briefs that could be instructive for practitioners seeking to develop or implement similar initiatives. Second, a comprehensive report of all findings will be written by the research team to inform DHHS, state and federal policymakers, and others of the implications of the findings for policy. The report will also include lessons for Oklahoma to refine the implementation of its initiative and to present recommendations for future evaluation of the OMI. The administrative data and reports, and the data and information collected through the semi-structured interviews and focus groups will be incorporated into these publications as noted below:
Practitioner Briefs (Summer 2006-Fall 2007). Up to 11 practitioner briefs will be produced based on the findings of this study. The first two briefs will draw on information gathered during informational interviews with senior OMI and PSI officials during the early planning stages of this project as well as information and findings from existing documents and reports prepared prior to this study. The remaining briefs will rely on the results of the semi-structured interviews and focus groups for which we are requesting clearance, as well as other sources of data such as direct observations and reports from the program’s administrative database.
Final Report (Spring-Summer 2007). The final report will be a comprehensive document laying out all the findings of the study, drawing upon and incorporating information from all project data sources mentioned above.
b. Project Schedule
See Table 5.
TABLE 5
PROJECT SCHEDULE
|
Timeframe |
Deliverable(s) |
Instrument Preparation |
February-July 2006 |
Draft and Final Interview Questionnaires Draft and Final OMB Clearance Package |
Obtain Documentary and MIS Data |
November 2005-September 2006 |
None |
Conduct Semi-Structured Interviews and Focus Groups |
February-May 2007 |
None |
17. Reasons for not Displaying Expiration Date for OMB Approval
The OMB number and expiration date will appear on the cover page of the protocols.
18. Exception to the Certification Statement
Exception to the certification statement is not requested.
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Respondent Universe and Sampling
The process study of the Oklahoma Marriage Initiative relies primarily on qualitative data for which the study team needs to obtain information from individuals who are most knowledgeable about the initiative. When identifying informants, it is necessary to focus on the potential universe of respondents and determine whether “purposeful identification” of respondents or a representative approach is most appropriate. Purposeful selection is most appropriate when insights and information can come only from individuals with particular roles or knowledge, such as lead staff at public agencies that collaborate with the Oklahoma Marriage Initiative. In contrast, a “representative” approach generally is most relevant for examining responses of workshop leaders and participants when there is a large number in the universe. Table 6 summarizes our approach for the five types of respondents.
In total, we will interview representatives from the five public institutions that collaborate on the OMI and from the community/volunteer sector. Representatives will include public agency staff, workshop leaders, and workshop participants. The majority of data collection is focused on agencies and individuals who are actively involved in OMI services, particularly at the workshop leader and participant level. However, we also plan to interview lead staff at four additional agencies, including some that delivered services in the past, or who intended to deliver services but for various reasons were unable to reach a critical mass.
TABLE 6
SUMMARY OF SAMPLING APPROACH
Respondent Group |
Interview Type and Number |
Respondents or or Focus Group Participants |
Sampling Approach |
Lead staff at public agencies |
Semi-structured individual or small-group interviews (11) |
Education State Career Tech oversight staff (1) Cooperative Extension directors (1) Head Start lead staff (1)
Social services TANF agency directors (1) Adoptive/foster parent unit leaders (1)
Corrections Prison wardens or unit leaders (3) OAYS unit leaders/directors (1)
Health Child guidance unit leaders (1)
Military Military base commanders or chaplains (1)
|
Purposeful Identification: MPR obtains names and contact information from PSI and arranges individual or small group interviews. |
Workshop leaders at public agencies |
Group interviews (7) |
Education High school teachers (1)
Social services Within My Reach trainers (1) Adoptive/foster parent staff (1)
Corrections Prison chaplains (3) OAYS staff (1)
|
Representative Approach: For
each selected agency,
|
Workshops leaders in community/ volunteer sector |
Group Interviews (2) |
Community/volunteers Faith leaders/clergy (1) Mental health counselors (1) |
Representative Approach: MPR identifies trained workshop leaders through OMI’s MIS, stratified by activity level. MPR over-recruits at twice the desired attendance, ensuring representation from each stratum.
|
W
TABLE
6 (continued) |
Focus groups (7) |
Education High school students (1)
Social services TANF recipients (1) Adoptive/foster parents (1)
Corrections Prison inmates (3) Families of youth first offenders (1) |
Representative Approach: MPR obtains list of participants from agency and randomly orders the list. If agency can provide contact information without compromising confidentiality, MPR recruits for focus groups. If not possible, MPR provides randomly ordered list to agency staff and enlists their assistance in recruiting. Participants over-recruited by 50 percent, and offered a $35 incentive.
|
Workshop participants in community/volunteer sector |
Focus groups (2) |
Members of faith community (1) General public (1) |
Representative Approach: MPR develops a recruitment flyer for focus groups. MPR and OMI enlist assistance of all trained workshop leaders in OKC and Tulsa to distribute flyers between May and October 2006. Participants call into a toll free number if interested. MPR recruits from this list when time comes for focus groups. Participants recruited at twice the required attendance and offered a $35 incentive.
|
Lead Public Agency Staff Representing OMI Sectors. Over time the initiative has collaborated with public agency staff in five institutional sectors (Education, Social Services, Corrections, Health, and Military). The number of collaborating public agencies varies within each institutional sector. In total, we plan to engage the public agencies through individual or group semi-structured interviews with lead staff from eleven collaborating agencies, three of which will be adult correctional facilities where Prison PREP is in full operation. For these semi-structured interviews, the study team will use a purposeful approach in selecting respondents, since their roles enable them to contribute unique insights and information. We will identify all relevant staff for semi-structured interviews and schedule appointments for in-person discussions with the assistance of OMI leadership. To learn about the barriers to implementing a marriage education program in the prison setting, we will also conduct telephone interviews with wardens and lead staff at prisons that have so far been less effective in implementing Prison PREP.
Workshop Leaders at Public Agencies. We have selected three of the five institutional sectors that have collaborated with the OMI for further investigation—Education, Social Services, and Corrections—including a particular emphasis on the latter. For each public agency within these three institutions (high schools, TANF agencies, Oklahoma's adoptive/foster care agency, Oklahoma state prison facilities, and the Oklahoma Association of Youth Services), we will identify a sample of workshop leaders for the group interviews. Based on the number of staff who have completed training, provided by PSI, we estimate the universe of public agency workshop leaders to be approximately 800 individuals. From this universe, we will conduct seven group interviews with up to 12 workshop leaders each (84 individuals). Three of the seven group interviews will be held with prison chaplains or other staff functioning as Prison PREP workshop leaders (up to 36 leaders).
Workshop Leaders in the Community/Volunteer Sector. Again using data provided by PSI, we estimate the universe of trained workshop leaders from the community/volunteer sector to be approximately 1,500 individuals. From this universe, we will select up to 12 workshop leaders for each of two group interviews, for a total maximum of 24 individuals.
Workshop Participants at Public Agencies. To the extent data are available, we will identify the total number of individuals who have participated in the OMI workshops at public agencies, from the OMI’s database. Seven focus groups will be conducted with up to 12 workshop participants from the same five public agencies where we convene semi-structured interviews with workshop leaders, for a total maximum of 84 individuals. Three of the seven focus groups with participants will be held with prisoners or ex-offenders from the three correctional facilities we will visit.
Workshop Participants in Community/Volunteer Sector. To the extent that data are available, we will determine the total number of individuals who have participated in OMI workshops in the community sector. We will conduct a focus group with up to 12 individuals who have participated in workshops within the faith community, and another focus group with up to 12 individuals who participated within a mental health or counseling context, for a total maximum of 24 participants.
2. Procedures for the Collection of Information
Table 6 identifies the intended informants for data collection and the number of semi-structured interviews and focus groups to be conducted. Semi-structured interviews or focus groups will be scheduled with staff from public agencies representing OMI sectors, workshop leaders at public agencies and from the community/volunteer sector, and workshop participants at public agencies and from the community/volunteer sector. The planned procedure for collecting data from each type of respondent is described below.
Lead Public Agency Staff Representing OMI Sectors. We will obtain contact information for the relevant lead staff at nine public agencies corresponding to the OMI’s five selected public institutions (Education, Social Services, Corrections, Health, and Military). These interviews will be conducted in person and scheduled as group or individual interviews, based on the topics being covered and the relevant experiences of the individuals being interviewed.
Workshop Leaders at Public Agencies. We will conduct up to seven semi-structured group interviews with workshop leaders at public agencies associated with three of the public institutions, selected because of the extent of activity in each. These include Education, Social Services, and Corrections. To contact institutional workshop leaders, we will first identify the trained leaders included in a database maintained by OMI staff, grouped by the number of workshops they have completed. We will consult with lead public agency staff prior to selecting participants to verify and update information for workshop leaders. If the number of trained workshop leaders who have conducted workshops within a given institutional sector is substantially more than can be accommodated in our data collection schedule, we will randomly sample from the overall list of potential respondents. Once we identify the list of potential respondents, we will work with agency staff to over-recruit by 50 percent (if possible), and schedule the interviews.
Workshop Leaders in the Community/Volunteer Sector. We will conduct two focus groups with the volunteer workshop leaders from areas that have been the most active: the faith area and the mental health/counseling area. Scheduling focus groups with the community/volunteer workshop leaders will require significant creativity due to the volunteer nature of their commitment to the OMI. We plan to obtain lists from the OMI of all trained volunteer workshop leaders, the timing of their training, and the number of workshops they have reportedly conducted. We will stratify the lists based on whether or not workshops have been conducted, with the aim of ensuring that 2/3 of each group is composed of leaders who have conducted workshops. After stratification, we will randomly order the list and recruit the desired number of participants, ensuring representation from each stratum. Because the volunteer workshop leaders are less likely to be connected to a public agency that is a full OMI partner, we will attempt to recruit twice the desired attendance at these focus groups. For example, we will likely aim to conduct a group interview with 12 workshop leaders from the community, where two-thirds (or 8) of the participants have conducted workshops and one-third (or 4) have not. To ensure the desired level of participation at each group, we will recruit 24 participants, including 16 who have provided a workshop and 8 who have not.
During telephone calls to recruit workshop leaders, MPR will conduct a short screening interview so that we can identify any obvious differences between those trained leaders who agree to participate in the focus group and those who do not. This screener will include up-to-date contact information, the agency or organizational affiliation for the leader, whether the leader ever conducted a PREP workshop, and the major obstacles to conducting workshops if none has been conducted.
Workshop Participants at Public Agencies. We will conduct up to seven focus groups with participants in the same public agencies where we will gather information from workshop leaders. In recruiting for these groups, we anticipate that lead agency staff will be able to generate a list of past participants. Depending on the size of the list, we may ask agency staff to compile a list of participants for a specified time period only. After receiving the list, we will order it randomly to ensure that known or readily accessible participants are not targeted. If agency staff can provide contact information for participants without compromising confidentiality, MPR will recruit individuals for the focus groups. However, if agency staff are unable to provide contact information, MPR will seek their assistance to recruit for the group; in these cases, we will provide the list of participants to agency staff in segments to avoid selection of the most readily accessible participants. Recognizing the likelihood of a high no-show rate, we will over-recruit by at least 50 percent.
Workshop Participants in Community/Volunteer Sector. We will conduct two focus groups with participants; one with members of a faith community and one with participants in a mental health or counseling setting. In the community/volunteer sectors, there are no centralized and readily accessible lists of participants and it is likely that workshop leaders do not maintain complete records of participants. Consequently, we plan to identify potential participants in these focus groups by seeking the assistance of all trained leaders in the community/volunteer sector who provide workshops in either the Oklahoma City or Tulsa area. We will work with the OMI to distribute recruitment flyers to all workshop participants by including flyers in the workshop materials sent to leaders by the OMI. The flyer will invite participants to contact MPR at a toll-free phone number to learn about the focus group and will describe incentives for participating. When a participant calls the number, an MPR staff person will thank the individual for his or her interest in the focus group, gather contact information, describe the incentives provided for participating in the focus group, and inform the person about the tentative timeframe for focus groups (fall 2006). When it is time to schedule the focus group, we will use this list to recruit participants. Given the timeframe of several months, this method of recruitment should generate a list sufficient in size for conducting two focus groups with up to 12 participants in each. However, it will require substantial over-recruiting; MPR staff will attempt to recruit twice the desired attendance.
3. Methods to Maximize the Response Rate and to Deal with Nonresponse
Nonresponse is of particular concern for researchers using group interview and focus group methods. To address concerns with potential nonresponse, we will over-recruit participants for each scheduled group interview and focus group. In addition, MPR will provide each recruit with a follow-up letter, thanking them for their interest, and outlining all of the relevant information they will need to attend the group. On the day before each group, telephone calls will be placed to each recruit, politely reminding them of the day and time of the group, and asking them to contact us if they have an emergency and cannot attend. For the data collection to be held state correctional facilities, we will enlist the assistance of the warden and other lead staff to ensure the prisoners are available for focus groups and interviews.
4. Tests of Procedures and Methods to be Undertaken
The instrument to be used during each interview or focus group is semi-structured in design. Consequently, we will not pretest instruments.
5. Individuals Consulted on Statistical Aspects of the Design
This study is being conducted by Mathematica Policy Research, Inc., under contract to the Office of the Assistance Secretary for Planning and Evaluation, U.S. Department of Health and Human Services. The project director is Ms. Robin Dion and the principal investigator is Mr. Alan Hershey, both of MPR. The project team consulted with Dr. Barbara Devaney and Dr. Sameena Salvucci of MPR about the sampling approach for the process evaluation. The analysis will not employ statistical methods, since the data are qualitative in nature and collected through the semi-structured interviews and focus groups.
APPENDIX A
AUTHORIZING LEGISLATION
Section 301 of the Public Health Service Act (42 U.S.C.241)
TITLE 42 > CHAPTER 6A > SUBCHAPTER II > Part A > § 241
§ 241. Research and investigations generally
(a) Authority of Secretary
The Secretary shall conduct in the Service, and encourage, cooperate with, and render assistance to other appropriate public authorities, scientific institutions, and scientists in the conduct of, and promote the coordination of, research, investigations, experiments, demonstrations, and studies relating to the causes, diagnosis, treatment, control, and prevention of physical and mental diseases and impairments of man, including water purification, sewage treatment, and pollution of lakes and streams. In carrying out the foregoing the Secretary is authorized to—
(1) collect and make available through publications and other appropriate means, information as to, and the practical application of, such research and other activities;
(2) make available research facilities of the Service to appropriate public authorities, and to health officials and scientists engaged in special study;
(3) make grants-in-aid to universities, hospitals, laboratories, and other public or private institutions, and to individuals for such research projects as are recommended by the advisory council to the entity of the Department supporting such projects and make, upon recommendation of the advisory council to the appropriate entity of the Department, grants-in-aid to public or nonprofit universities, hospitals, laboratories, and other institutions for the general support of their research;
(4) secure from time to time and for such periods as he deems advisable, the assistance and advice of experts, scholars, and consultants from the United States or abroad;
(5) for purposes of study, admit and treat at institutions, hospitals, and stations of the Service, persons not otherwise eligible for such treatment;
(6) make available, to health officials, scientists, and appropriate public and other nonprofit institutions and organizations, technical advice and assistance on the application of statistical methods to experiments, studies, and surveys in health and medical fields;
(7) enter into contracts, including contracts for research in accordance with and subject to the provisions of law applicable to contracts entered into by the military departments under sections 2353 and 2354 of title 10, except that determination, approval, and certification required thereby shall be by the Secretary of Health and Human Services; and
(8) adopt, upon recommendations of the advisory councils to the appropriate entities of the Department or, with respect to mental health, the National Advisory Mental Health Council, such additional means as the Secretary considers necessary or appropriate to carry out the purposes of this section.
The Secretary may make available to individuals and entities, for biomedical and behavioral research, substances and living organisms. Such substances and organisms shall be made available under such terms and conditions (including payment for them) as the Secretary determines appropriate.
(b) Testing for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects; consultation
(1) The Secretary shall conduct and may support through grants and contracts studies and testing of substances for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects. In carrying out this paragraph, the Secretary shall consult with entities of the Federal Government, outside of the Department of Health and Human Services, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct for such entity studies and testing of substances for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects.
(2)
(A) The Secretary shall establish a comprehensive program of research into the biological effects of low-level ionizing radiation under which program the Secretary shall conduct such research and may support such research by others through grants and contracts.
(B) The Secretary shall conduct a comprehensive review of Federal programs of research on the biological effects of ionizing radiation.
(3) The Secretary shall conduct and may support through grants and contracts research and studies on human nutrition, with particular emphasis on the role of nutrition in the prevention and treatment of disease and on the maintenance and promotion of health, and programs for the dissemination of information respecting human nutrition to health professionals and the public. In carrying out activities under this paragraph, the Secretary shall provide for the coordination of such of these activities as are performed by the different divisions within the Department of Health and Human Services and shall consult with entities of the Federal Government, outside of the Department of Health and Human Services, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct and support such activities for such entity.
(4) The Secretary shall publish a biennial report which contains—
(A) a list of all substances
(i) which either are known to be carcinogens or may reasonably be anticipated to be carcinogens and
(ii) to which a significant number of persons residing in the United States are exposed;
(B) information concerning the nature of such exposure and the estimated number of persons exposed to such substances;
(C) a statement identifying
(i) each substance contained in the list under subparagraph (A) for which no effluent, ambient, or exposure standard has been established by a Federal agency, and
(ii) for each effluent, ambient, or exposure standard established by a Federal agency with respect to a substance contained in the list under subparagraph (A), the extent to which, on the basis of available medical, scientific, or other data, such standard, and the implementation of such standard by the agency, decreases the risk to public health from exposure to the substance; and
(D) a description of (i) each request received during the year involved—
(I) from a Federal agency outside the Department of Health and Human Services for the Secretary, or
(II) from an entity within the Department of Health and Human Services to any other entity within the Department,
to conduct research into, or testing for, the carcinogenicity of substances or to provide information described in clause (ii) of subparagraph (C), and (ii) how the Secretary and each such other entity, respectively, have responded to each such request.
(5) The authority of the Secretary to enter into any contract for the conduct of any study, testing, program, research, or review, or assessment under this subsection shall be effective for any fiscal year only to such extent or in such amounts as are provided in advance in appropriation Acts.
(c)
Diseases not significantly occurring in United States
The
Secretary may conduct biomedical research, directly or through grants
or contracts, for the identification, control, treatment, and
prevention of diseases (including tropical diseases) which do not
occur to a significant extent in the United States.
(d)
Protection of privacy of individuals who are research subjects
The
Secretary may authorize persons engaged in biomedical, behavioral,
clinical, or other research (including research on mental health,
including research on the use and effect of alcohol and other
psychoactive drugs) to protect the privacy of individuals who are the
subject of such research by withholding from all persons not
connected with the conduct of such research the names or other
identifying characteristics of such individuals. Persons so
authorized to protect the privacy of such individuals may not be
compelled in any Federal, State, or local civil, criminal,
administrative, legislative, or other proceedings to identify such
individuals.
APPENDIX B
GUIDE FOR SEMI-STRUCTURED INTERVIEWS WITH LEAD STAFF
AT PUBLIC AGENCIES
Oklahoma Marriage Initiative Process Evaluation
Guide for semi-structured interviews with
lead
staff at public agenicies
This protocol will be used to guide semi-structured individual or small-group interviews with current and/or former staff at public agencies in selected locations that sponsor or considered sponsoring OMI relationship skills workshops and related activities, either currently or in the past, or who have integrated OMI-related content into existing curricula. Each interview is expected to last about 90 minutes. Only relevant sections of the protocol will be administered to each group of respondents. Interviews will be conducted with the relevant lead staff at the following Oklahoma public agencies:
Department of Corrections
Department of Human Services Local TANF agencies
Department of Human Services Adoptive/Foster Parent units
Department of Education, Division of Career Technology
Oklahoma Association of Youth Services First Time Offenders Program
Local military installations
Oklahoma State University, Cooperative Extension Services
Department of Health Child Guidance division
Community Action or Head Start agencies
A. learning of the omi
How did you or your agency first learn about the OMI? Who approached your agency to solicit your interest in OMI, and when?
Is your agency currently involved in any way with OMI, such as providing referrals or currently using OMI curriculum materials or messages?
If so, what is the nature of you or your agency’s current involvement with the OMI or its programs and services, or your use of the PREP curriculum?
How long have you been involved with the OMI in this way?
Initially, what appealed to you about the OMI or PREP-related services? What concerned you? How were thee concerns addressed?
How do the goals of the OMI fit with your agency’s mission?
B. reasons AGENCY not ACTIVELY INVOLVED IN OMI (IF APPLICABLE)
What types of activities or programs related to OMI or the PREP curriculum have you considered or been asked to implement?
What was your agency’s process for exploring or considering these activities or programs?
What issues or factors have you considered in deciding whether or how to become involved in OMI? Which were decisive, and why?
Relevance or appropriateness of addressing marriage with agency clients
Access to interested participants
“Fit” with existing mission, program, or services
Availability or lack of resources
Do you currently support OMI-related activities in other ways?
Do you think your agency may take on any type of role or activity related to OMI in the future? What types of roles or activities would fit well with your agency’s agenda and structure?
c. background
Briefly describe the OMI-related program or activity(ies) that you oversee or that you helped design or implement, including their purpose, content, structure, and locations.
Do you coordinate with any other agency, non-profit, or other entities to provide this program or service (other than PSI)?
Why and how did your agency decide to offer this special program or service? Who initiated the idea of creating the program?
Why was your agency interested in the OMI or in providing OMI-related services?
What benefits did you anticipate for your agency or clients?
What concerns, if any, did you have?
To whom is the OMI-related program or activity offered or targeted?
Are there specific eligibility criteria that clients must meet to participate in the program or activity?
Why is this group a focus of the program or activities?
Is participation voluntary or mandatory?
What was the process for planning and implementing the OMI activity or service?
Does your agency have any formal or informal agreements with PSI or DHS regarding your OMI-related activity?
Can clients at your agency receive other services related to marriage and relationships besides the PREP workshops? If so, please describe these other services.
Does the agency provide OMI-related services at multiple locations? If so, please specify the number of locations.
How many total locations does this agency have in Oklahoma (e.g., number of prisons, high schools, TANF agencies)?
How many total full-time employees are devoted to operating the OMI-related program at each participating location of your agency?
What proportion of your total staff is this?
Who is the staff contact person at PSI with whom you coordinate services or activities?
NOTE TO INTERVIEWER:
Administer Section D to lead staff at agencies that deliver OMI workshops. For agencies that embed PREP material into a broader program or curriculum, skip to Section E.
D. CURRICULUM and training (STAND ALONE WORKSHOPS)
What curriculum is used to deliver the OMI workshop? Is this an adaptation of PREP?
If so, how was the need for adaptation determined?
What adaptations were made to work with the targeted population?
Who created these adaptations, and what was the process for creating them?
Did you have a role in the curriculum adaptation? If so, what was that role?
What is the frequency and duration of the workshop? How many total hours of curriculum are provided over what period of time?
How are workshop leaders identified and recruited?
Are the workshop leaders agency staff? PSI staff? Independent contract workers?
Who performs the recruiting function—PSI staff or agency staff?
What kind of background or experience do agency workshop leaders typically have?
Who provides the agency’s workshop leaders with training in how to use the curriculum with agency clients?
The curriculum authors? Agency staff? PSI staff?
What happens when new staff come on board and need training?
How many agency staff members have been trained to conduct workshops?
How many actively conduct workshops?
Does the agency plan to train more in the future?
Is the curriculum delivery supervised or monitored by agency staff, PSI, or others?
If so, who provides this supervision?
What does this supervision entail?
Is assistance or support available from others outside of your agency in arranging for or conducting workshops?
If so, who provides this assistance or support?
What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?
Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance?
Is the issue of domestic violence relevant to the program, class, or activity you provide?
How have you addressed it in your planning?
How do you address it in your operations?
E. CURRICULUM and training (broader programs)
What overall curriculum is used in your program?
How does PREP fit into the purpose of your program, and your existing or previous curriculum?
Why did you decide to incorporate PREP into your program or class? How was the decision made, and who made it? How was PSI involved?
Do you use an adaptation of PREP, or was your original curriculum adapted to incorporate PREP elements? Or both? If so, why was adaptation needed?
What kinds of adaptations were made? (e.g. deletion of certain segments or elements)
Who created these adaptations, and what was the process?
Did you have a role in the curriculum adaptation? If so, what was that role?
What is the frequency and duration of the course or program? How many total hours of curriculum is provided over what period of time?
What proportion of the broader course or program is devoted to PREP content (e.g. number of hours or sessions)?
Who trains your program or course teachers/facilitators on the PREP component?
What kind of background or experience, skills, or credentials do teachers/facilitators have?
How many teachers or facilitators have been trained to conduct workshops? How many actively conduct workshops?
Is instruction in the curriculum supervised or monitored by agency staff, PSI, or others?
If so, who provides this supervision?
What does this supervision entail?
Is assistance or support available from others outside of your agency in arranging for or conducting workshops?
If so, who provides this assistance or support?
What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?
Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance?
Is the issue of domestic violence relevant to the program, class, or activity you provide?
How have you addressed it in your planning?
How do you address it in your operations?
f. recruiting AND enrolling participants
Who has primary responsibility for recruiting or enrolling individuals in the OMI-related program or activity?
Have these staff received training in how to identify and recruit participants?
If so, what kind of training do they receive, who provides it, and for how long?
Are classes easy or hard to fill? Is there a waiting list for classes?
Are all possible participants informed of the service, or are only specific groups or individuals targeted?
If so, who is targeted and how are the individuals identified?
How are individuals recruited or enrolled for the activity?
What is the context or setting of the recruitment effort?
For example, a one-on-one meeting, a group orientation session, a visit in the home? Is an invitation sent through the mail? Is the program presented to both partners if client is married? To youth and their parents?
Is the recruitment “occasion” one that is exclusively for discussing and enrolling the client in the program, or is it integrated into some other purpose for client contact? If integrated, how well does this work?
What “messages” are potential participants given about the activity?
What are they told of the purpose of the activity or program; what they may gain from it; what the level of commitment to participation is?
Are the potential recruits offered any incentives or additional agency benefits or credits for participating? If so, please describe these.
g. client participation and reactions
How long has your organization been providing the OMI-related program or activity?
Does your agency track and maintain program management data beyond what is reported to the OMI?
If so, what types of information are tracked?
How are the records kept (e.g., electronic or paper), and who maintains them?
How many curriculum workshops has the agency provided to date?
How many individuals at your agency have participated in the workshop, program or activity to date?
If participation is open to anyone, who typically enrolls or participates? Couples? Singles? Youth? Girls/Women? Boys/Men? Parents?
Do you track referrals to your program? How many total individuals have been referred to your program or workshop? Which organizations are responsible for most referrals?
Is activity by workshop leaders tracked by location? Is it recorded in the OMI’s database, or elsewhere?
How do participants generally respond to the curriculum workshops?
In what ways does the material interest them, or meet their needs?
What gaps or shortcomings have you observed in the curriculum or activities?
What is the general attitude among your client population regarding marriage?
Have you noticed a change in general attitudes since you began offering the OMI-related program or activity?
What factors besides marriage and relationship skills and knowledge do you believe affect or will affect the decisions your clients make or will make about marriage and relationships?
h. Systems and Culture Change
What was the agency’s culture with respect to marriage and relationships before it began offering OMI-related services?
Did staff discuss relationships or marriage with agency clients?
Were any services offered to help agency clients with their relationships?
Was there any formal or informal policy with respect to addressing the topic of clients’ relationships or marriage?
Has the agency specifically sought to create change in its culture regarding marriage?
For example, has it sought to raise awareness of the importance of relationships and marriage for client’s well-being? If so, how was this done?
In what other ways has it sought to create change in agency culture?
Has the process of being trained in the OMI-related services affected the attitudes or behavior of staff members who provide referrals or deliver OMI-related materials or workshops?
Have you noted changes in staff attitudes or the way they approach their clients with regard to marriage and relationships?
For example, changes in attitudes toward working with fathers or couples? About clients’ potential for healthy marriage? About value of marriage education?
If not, do you think there will be any such changes in the future?
Has the OMI-related program affected the operations or structure of the agency or the location at which the activities are provided? If so, how?
If not, do you think there will be any such changes in the future?
Has the content of the program or curriculum affected how members of your organization work together, handle problems, or communicate? If so, how?
i. Wrap-Up
What characteristics, elements, or implementation approaches of the OMI-related service changed over time, if any? Why were these changes made?
In general, what organizational characteristics or strategies do you think are important in successfully developing and implementing classes, programs, or services related to OMI?
What are the greatest challenges, and the most important lessons your agency has learned in making these classes, services, or programs available?
APPENDIX C
GUIDES FOR GROUP INTERVIEWS WITH WORKSHOP LEADERS
Guide for Group Interviews with Workshop Leaders at Public Agencies
Guide for Group Interviews with Community-Volunteer Workshop Leaders
Oklahoma Marriage Initiative Process Evaluation
guide for group
interviews with public
agencY WORKSHOP LEADERS
This protocol will be used to guide semi-structured small-group interviews with a sample of workshop, course, or program leaders who deliver OMI-related services at public agencies or institutions. Five group interviews will be conducted with up to 12 curriculum instructors each at selected locations. Each interview will last about 90 minutes.
TANF agencies: Within My Reach trainers
Child welfare agencies: Adoptive/foster parent staff
Department of Corrections prison facilities: Prison chaplains
Oklahoma Association of Youth Services first-time offender program facilitators
High school Family and Consumer Sciences teachers
A. Introduction (10 min)
Introduce self and other MPR Staff
Purpose of discussion
Instruction in relationship skills is being offered by various state agencies in Oklahoma, stimulated by or as a part of the Oklahoma Marriage Initiative. We are interested in learning about your experiences conducting these workshops/classes so that we can understand better how they are delivered in various settings.
During the discussion today we would like to hear about your experiences delivering OMI workshops/PREP curriculum materials for your agency. Specifically, we would like to hear about the training you received to deliver this material, how your agency recruits or enrolls participants for workshops/classes, your experiences using the curriculum, and whether and how delivering the workshops/classes/materials may have affected participants or yourselves, or resulted in any changes at your agency.
Confidentiality and audiotaping
We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”
We will be audiotaping to help us summarize the discussion later, but no one will be given audiotapes outside of MPR
To help ensure we have a clear tape, we ask that only one person talk at a time
My colleague may also be taking some notes during the discussion
Time restrictions—moderator must move discussion to all topics
Offer opinion even if different from others; no right or wrong answers
B. background (10 min)
How long have you been involved with providing [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] for [PUBLIC AGENCY OR INSTITUTION]?
Why and how did you first become involved with delivering this workshop/class or using this material?
Did you have a choice in whether or not you would deliver this workshop/class or use the material?
What initially appealed to you about it? What concerns did you have, if any? How did these change over time?
How well did delivering OMI-related instruction fit with your personal goals and beliefs?
How often do you present the curriculum material? Do you provide instruction on a regular basis, or intermittently?
On average, how much time do you spend preparing for and delivering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] in a typical month?
C. curriculum training and ongoing assistance (15 min)
Tell me about the training you received to deliver [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS].
Who conducted the training? Where did it occur? How long did it last?
Was the training specifically for your agency’s staff, or did it include people from other agencies or organizations?
Was it tailored specifically for the curriculum adaptation you use, or was the training focused on the standard PREP program?
How prepared did you feel to deliver [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] workshops after this initial training?
Did you feel prepared to organize workshops, including where and when you would offer them?
Did you feel prepared to recruit participants? Did you know where or how you would recruit them?
Have you requested or received any refresher training? Do you think this would be useful?
Is your instruction in the curriculum supervised or monitored by agency staff, PSI, or others?
If so, who provides this supervision?
What does this supervision entail?
Is assistance or support available from others outside of [AGENCY OR INSTITUTION] in arranging for or conducting workshops?
If so, who provides this assistance or support?
What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?
Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance?
D. CURRICULUM ADAPTATION and format (15 min)
To your knowledge, were any formal adaptations made to the original PREP curriculum for use with the participants served by your agency?
If so, who made these adaptations, what adaptations were made, and why?
Did you or other agency staff have any role in the adaptation process? What was that role? For example, did you or others provide feedback to the curriculum authors, participate in a focus group, permit authors to observe classes?
To your knowledge, were any formal adaptations made to your original program or course curriculum when the PREP content was added? (if applicable)
If so, who made these adaptations, what adaptations were made, and why?
Did you or other agency staff have a role in the adaptation process? What was that role? For example, did you or others provide feedback to the curriculum authors, participate in a focus group, or permit authors to observe your classes?
Many instructors make minor changes to curriculum content or format during its delivery to make it more relevant to or appropriate for their audience, class, or participants.
Other than the formal adaptations we discussed before, do you typically make other changes when delivering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
If so, what are these changes and why are they needed?
Let’s talk about the format you use for providing [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS].
How many total hours of instruction are provided, over how many days/weeks/months?
Why did you choose this format over other formats?
What proportion of the program or class is devoted to the PREP material on relationships and marriage?
E. recruiting and enrolling participants (15 min)
Who has primary responsibility for identifying and recruiting participants for the [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
For example, agency staff such as caseworkers, teachers, high school counselors, or chaplains? Or is this the responsibility of the curriculum instructor?
Is [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] mandated for certain clients (required by the agency to fulfill a requirement) or court-ordered?
Are there any eligibility criteria that clients must meet to participate in the program? Or is the program open to any agency client that wishes to participate?
What are the eligibility requirements, if any? Why were these requirements identified?
Do clients receive special benefits, incentives, or credits for participating in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]? If so, please describe.
Does your agency have trouble filling slots for [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS], or are they usually filled to capacity?
What strategies do you or your agency use to identify and recruit participants for [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
What strategies appear to be most successful at recruiting participants?
Which strategies do not appear to work well?
What would help your agency recruit and enroll more participants?
Have you (or other agency staff) received any training in conducting outreach, marketing the workshop, or identifying and recruiting participants for this program/class?
If so, who provided the training, when, and how long did it last?
F. CLIENT participation and reactions (15 min)
1. Who typically attends the [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
Couples? Singles? Youth? Women/Girls? Men/Boys? Parents?
Do participants usually represent a cross-section of [AGENCY OR INSTITUTION], or do specific types of individuals or couples tend to be attracted?
2. Based on your experiences, how well do you think [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] meets the needs of your agency’s clients?
How interested do clients appear to be in learning the curriculum skills and information?
How well do clients seem to understand and apply the curriculum content?
What reactions do your agency’s clients have to participating in a group workshop that teaches relationship and marriage skills?
What element(s) of the group workshop do participants appear to enjoy the most?
Are there aspects of the curriculum that participants seem to dislike? What are these aspects?
How is attendance at [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
Have you experienced frequent absences from individual classes or sessions? If so, to what do you attribute these absences?
What level of attrition (dropping out prior to the conclusion of all sessions/classes) do you tend to see?
Can you estimate the percent of participants that tend to complete the workshop?
At what point in the program does attrition tend to occur?
Has the issue of domestic violence or dating violence come up as part of providing [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
If so, how frequently does it come up? How do you deal with it?
Have you received any special training on recognizing or dealing with domestic or dating violence? How adequate was this training, in your view?
g. Systems and CULTURE CHANGE (15 min)
One of the OMI’s goals is to change the way Oklahomans think about marriage and divorce, both at the level of individual citizens and in the way public and private organizations and agencies deal with individuals.
1. In your opinion, has offering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] as one of your agency’s services changed the way staff think about or treat their clients?
Before [AGENCY OR INSTITUTION] began offering instruction in relationship skills, what was the general agency culture with respect to marriage and relationships?
Have staff attitudes about their clients changed? If so, how?
Have staff interactions with their clients changed? If so, how?
Has offering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] changed the agency’s structure, operations, or policies?
In what ways has your agency’s structure, operations, or policies changed?
Has being trained in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] changed the way you or other staff work together, work with participants, or handle conflict?
Have your own attitudes or behavior regarding relationships and marriage changed as a result of receiving the PREP training and/or conducting workshops?
Have you noticed any changes in participants’ attitudes toward marriage or divorce as a result of learning relationship skills in your workshops?
h. Wrap-up and post-group information form (5 min)
In your opinion, how well does the collaboration between PSI and your agency work?
What about the collaboration works well and what is not working well?
How could PSI and your agency work more collaboratively?
Are there other aspects of your role or experience as an OMI workshop leader for your agency that you would like to share with me?
COLLECT POST-GROUP INFORMATION FORM.
ATTACHMENT D
guide for group interviews with community volunteer
workshop leaders
Oklahoma Marriage Initiative Process Evaluation
guide for group interviews with community volunteer
workshop leaders
This protocol will be used to guide semi-structured small-group interviews with curriculum instructors from local communities who voluntarily provide PREP workshops to their clients or to members of the community. Two group interviews will be conducted with up to 12 workshop leaders; one with instructors in the faith sector; and another with instructors in the mental health, counseling or related fields. Each group interview will last about 90 minutes.
A. Introduction (10 min)
Introduce self and other MPR staff
Purpose of discussion
As you know, workshops or classes that provide information on relationship or marriage skills are being offered all across the state of Oklahoma, stimulated by or as a part of the Oklahoma Marriage Initiative. We are interested in learning about your experiences conducting these workshops/classes so that we can understand better how they are delivered in various settings.
Specifically, we would like to hear about the training you received to deliver this material, how you recruit or enroll participants for workshops/classes, your experiences using the curriculum, and whether and how delivering the workshops or classes may have affected participants or yourselves, or resulted in any changes in your communities.
Confidentiality and Taping
We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”
We will be audiotaping to help us summarize the discussion later, but no one will be given tapes outside of MPR
To help ensure we have a clear tape, we ask that only one person talk at a time
My colleague may also be taking some notes during the discussion
Time restrictions—moderator must move discussion to all topics
Offer opinion even if different from others; no right or wrong answers
B. BACKGROUND (5 min)
How long have you been involved with providing PREP workshops for the OMI?
How and when did you first hear about the OMI and the free PREP training it offers?
Why and how did you choose to become involved in delivering this workshop?
What made you interested in participating?
You thought it would be a valuable addition to other counseling services you offer?
You saw a need in your community or church for this kind of program?
You anticipated there would be benefits to your own relationship by learning PREP?
How important was the fact that the PREP training was free of charge in your decision to be trained to deliver workshops?
What initial concerns did you have about being trained or delivering the workshops? How were these concerns addressed?
C. training and ongoing assistance (15 min)
Tell me about the training you received to deliver PREP.
Who conducted the training? When and where did it occur? How long did it last? About how many people were in the training with you?
What did you like the best about the training? What was least useful, or could use improvement?
How prepared did you feel to deliver workshops after this initial training?
Did you feel prepared to organize workshops, including where and when you would offer them?
Did you feel prepared to recruit participants? Did you know where or how you would recruit them?
Have you requested or received any refresher training? Do you think this would be useful?
Is your delivery of PREP workshops supervised or monitored by anyone?
If so, who provides this supervision?
What does this supervision entail?
Is assistance or support available from others in arranging for or conducting workshops?
a. If so, who provides this assistance or support?
b. What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?
Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance? Did the assistance meet your needs?
Our understanding is that workshop leaders who receive free PREP training are requested to provide 4 workshops in their communities at no cost to participants. Have you conducted any workshops yet? If so, how many so far?
a. What made this obligation easy or difficult to fulfill?
b. Have you continued to provide workshops beyond the 4 required ones? If so, are they free? If not, what is the cost to participants?
If you have not yet fulfilled the obligation of providing four free workshops, why not?
a. Not enough time has elapsed since training
b. Enough time has elapsed but it has been difficult to recruit a sufficient number of participants to conduct four workshops
c. Other reasons
D. CURRICULUM ADAPTATION and format (10 min)
Many facilitators or instructors make minor changes to a curriculum so that it is more relevant to or appropriate for their audience, class, or participants.
What changes or refinements have you made when delivering PREP?
Why were these changes or refinements needed?
Are the modifications discussed with PSI or the curriculum authors?
Let’s talk about the format you use for providing the PREP workshop.
a. How many total hours of instruction are provided, over how many days/weeks/months?
b. Why did you choose this format over other formats?
Where have you conducted workshops?
a. Have you had trouble finding suitable locations/facilities?
b. Do you think the location has any effect on attendance (positive or negative)?
E. recruiting and enrolling participants (15 min)
To your knowledge, where do most of your participants first hear about your workshop?
Your agency? Your work? Your church? Your neighbors and friends? Word-of-mouth? The OMI website? From a flyer, poster or brochure? Networking with public agencies? Community event? Public service announcements? Other?
Which of these outreach strategies have you tried so far?
Which of them appear to be the most effective in generating interest in the workshop?
The least effective? Why?
Have you had trouble filling slots for a workshop? Or are workshops usually filled to capacity?
If any participants have been referred to you by other organizations, which organizations are they? Why and how did these sources agree to make referrals to you?
Have any participants in your workshops learned about you at a Sweetheart’s Weekend or other OMI-sponsored event?
Can participants receive or win gifts such as door prizes for participating in a workshop or hearing a presentation about it?
If so, what are these gifts and who funds them?
What role do you think such gifts or prizes play in encouraging enrollment and ongoing participation?
Have you ever received any training in conducting outreach, marketing the workshop, or identifying and recruiting participants for this program/class? If so, who provided the training, when, and how long did it last?
How could you be more successful planning workshops or recruiting participants? What kind of support or assistance would be helpful?
F. client participation and reactions (15 min)
1. Who typically attends your workshops?
Couples? Singles? Youth? Women/Girls? Men/Boys? Parents?
b. Do participants usually represent a cross-section of your community, or do specific types of individuals or couples tend to be attracted?
Based on your experiences, how well do you think the workshop meets the needs of your participants?
How interested do clients appear to be in learning the curriculum skills and information?
How well do clients seem to understand and apply the curriculum content?
How do participants in your community tend to react to the PREP workshop?
What element(s) of the workshop do they appear to enjoy the most?
Are there elements of the curriculum that participants seem to dislike? What are these aspects?
How is attendance at your workshops?
Have you experienced frequent absences from individual classes or sessions? If so, to what do you attribute these absences?
What level of attrition (dropping out prior to the conclusion of all sessions/classes) do you tend to see?
Can you estimate the percent of participants that complete all sessions?
At what point in the program does attrition tend to occur?
Has the issue of domestic violence or dating violence come up as part of providing the workshop?
If so, how frequently does this come up? How do you deal with it?
Have you received any special training on recognizing or dealing with domestic or dating violence? How adequate was this training, in your view?
g. community attitudes and norms about marriage (15 min)
One of the OMI’s goals is to change the way Oklahomans think about marriage and divorce.
Have your own attitudes or behavior regarding relationships and marriage changed as a result of receiving the PREP training and/or conducting workshops?
Have you noticed any changes in participants’ attitudes toward marriage or divorce as a result of learning relationship skills in your workshops?
If so, what kind of attitude changes have you observed?
What is the general attitude in your community toward marriage and divorce?
Besides having relationship and marriage skills, what other factors seem to affect people’s attitudes and norms about marriage and relationships in your community?
Do you expect that offering relationship skills classes in a community will gradually change the attitudes and norms about marriage? If so, how? If not, what would be needed to make this kind of community-level change?
h. Wrap-up and post-group information form (5 min)
Are there other aspects of your role as a community PREP workshop leader that you would like to share with us?
COLLECT POST-GROUP INFORMATION FORM.
APPENDIX E
GUIDES FOR FOCUS GROUPS WITH PROGRAM PARTICIPANTS
Guide for Focus Groups with Program Participants at Public Agencies
Guide for Focus Groups with Program Participants in Community
Oklahoma Marriage Initiative Process Evaluation
guide for focus groups with workshop participants at public agencies
This protocol will be used to guide focus groups with participants who receive OMI workshops as part of the services provided by a public agency or institution. Focus groups will be conducted with up to 12 participants each, at each of the five public agencies identified below. Each focus group will last about 90 minutes.
TANF recipients who participated in Within My Reach
Adoptive/foster parents who participated in a PREP-ENRICH retreat
Prisoners who participated in Prison PREP as part of a reentry program
High school students who took a Connections-PREP class
Youth and their parents who received a PREP class as part of a diversion program for first-time offenders
A. INTRODUCTION (5 MIN)
Introduce self and other MPR Staff
Purpose of Discussion
Relationship skills workshops or classes are being offered by various state agencies in Oklahoma, in coordination with the Oklahoma Marriage Initiative. We are interested in learning about your experiences participating in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] so that we can understand better how the program works.
During the discussion today we would like to hear about how you came to participate in this program, why you enrolled in it, what you thought of it, how useful it was in your daily life, and whether you think offering this program has changed the way the agency interacts with you.
Confidentiality and taping
We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”
We will be audiotaping to help us summarize the discussion later, but no one will be given tapes outside of MPR
To help ensure we have a clear tape, we ask that only one person talk at a time
My colleague may also be taking some notes during the discussion
Time restrictions—moderator must move discussion to all topics
Offer opinion even if different from others; no right or wrong answers
B. PARTICIPANT INTRODUCTIONS (10 MIN)
Please introduce yourself by telling us:
Your first name
When and where you attended the [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]
C. REFERRAL AND ENROLLMENT [NAME OF PREP ADAPTATION]
(15 MIN)
How did you come to participate or enroll in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
Was this class required to fulfill an agency requirement? For example, was it required as a condition of prisoner release, high school graduation, release from a first-offender program, or as a condition for adoption? If so, who first approached you about the class, and how was it described to you?
If the class was not required, was it recommended to you by an agency representative? For example, was it recommended by a school counselor or advisor, a caseworker, a prison chaplain, or a representative of the Oklahoma Marriage Initiative? If so, how was the class described to you?
If the class was not required and you did not hear about it from an agency representative, did you seek out the program as the result of seeing a flyer, poster, or hearing about it from a friend or acquaintance?
What were your reasons for participating in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
What interested you? What benefits or knowledge did you expect to get out of it?
What concerns did you have, if any? How were these addressed?
Were you eligible to receive any special benefits, like gift cards or door prizes, by participating in the program?
Were you eligible for any other agency benefits or services because of your participation in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?
What effect did these potential benefits have on your motivation to join or continue participating in the program?
NOTE TO INTERVIEWER: Administer Section D to focus groups who participated in an OMI workshop. For focus groups who received PREP material within a broader program or curriculum, skip to Section E. (e.g., high school students and first offenders).
D. EXPERIENCES WITH workshop (20 MIN)
Describe the format of a typical workshop/class meeting for me.
How large was your group? How often did you meet, and for how long at a time? Where were the meetings or classes held?
What topics did you discuss during your sessions?
Which topics seemed the most useful for you?
The least useful?
What kind of activities did you do during a typical class or workshop meeting? Did you listen to lectures? Have group discussions? Do written exercises? Watch videos?
Which activities did you like most?
Least?
Did any of your classes or sessions address issues of domestic violence, or dating violence, or other types of violence in relationships?
If so, what was said about violence in relationships?
How effective was your workshop/class instructor?
Was he/she able to convey the information clearly and in a way that you could clearly understand?
Was the material presented in a way that kept your interest and attention? Or were you often bored?
Were examples and illustrations relevant to your situation and life?
In what areas might the instructor improve? In what areas did he/she excel?
How difficult or easy was it for you to understand and apply the information and skills covered in the class?
Did you attend by yourself, with your spouse or partner, with your parents, or with someone else?
How many of the workshop/class meetings did you attend?
If you missed any sessions, what were the reasons you missed them?
If you dropped out of the program after some period, please explain when and why you dropped out.
Would you recommend this class/program/workshop to your friends and family? Why or why not?
E. EXPERIENCES of PREP material within a broader program (15 MIN) [ONLY FOR PROGRAMS USING PREP AS ONE COMPONENT OF A CLASS OR BROADER CURRICULUM]
You are attending this focus group because you participated in a class that involved some discussion about the topic of relationships and relationship skills.
About how much of the overall program was devoted to this general topic?
What kinds of topics were discussed during that session or module?
Which seemed to be most useful?
Least useful?
What kinds of relationships were addressed?
Relationships between you and your parents? Your friends? Boyfriends or girlfriends? Dating partners? Others?
Was marriage discussed? What did the program say about marriage?
How interested were you in this part of the course or program?
What interested you about it?
Did you find any parts of it boring? If so, why?
Were there topics, issues, or problems that you think should have been discussed during this class or module, but were not? If so, what were they?
How effective was your teacher or facilitator in presenting this class or module?
Was he/she able to convey the information clearly and in a way that you could clearly understand?
Was the material presented in a way that kept your interest and attention?
Were examples and illustrations relevant to your situation and life?
Would you recommend this class or module to your friends? Why or why not?
f. participation and reactions (15 MIN)
What is the main thing you learned from this workshop/class?
Have you used any of the skills or lessons in your daily life? If so, which skills?
What is the current attitude toward marriage and relationships among your friends, or in your neighborhood or community?
Have your attitudes or thoughts about relationships or marriage changed as a result of this workshop/class? If so, what were these changes?
Do you think the workshops/classes have helped you personally?
If so, how have they helped? How important would you say the workshop/classes have been in your life?
Have the workshop/classes improved your relationship with a significant other or with other family members? If so, how have they improved it?
What do you think works well in this program?
What changes would you suggest to improve the services?
g. WRAP-UP and post-group information form (10 MIN)
If you are in a relationship with a partner or spouse, are your instructors or agency staff, such as caseworkers, aware of that relationship?
Do you feel they are supportive of your relationship?
In what way have they been supportive or not supportive?
Is there anything else you’d like to tell me about workshop/class or your experience?
COLLECT POST-GROUP INFORMATION FORM.
PAY PARTICIPANTS AS THEY LEAVE (SIGN RECEIPT LOG)
appendix f
guide for focus groups with community workshop participants
Oklahoma Marriage Initiative Process Evaluation
guide for focus groups with community workshop participants
This protocol will be used to guide focus groups with individuals who have participated in OMI workshops provided by community volunteers. One group will focus on members of a faith community; a second focus group will be composed of individuals served by mental health/counseling providers and possibly other members of the general public. Up to 12 individuals will participate in each focus group, and each will last approximately 90 minutes.
A. INTRODUCTION (5 MIN)
Introduce self and other MPR Staff
Purpose of discussion
Relationship skills workshops or classes are being offered by various people all across Oklahoma, in coordination with the Oklahoma Marriage Initiative. We are interested in learning about your experiences participating in the PREP workshop so that we can understand better how the program works.
During the discussion today we would like to hear about how you came to participate in this program, why you enrolled in it, what you thought of it, how useful it was in your daily life, and whether you think offering this program has changed your relationships.
Confidentiality and taping
We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”
We will be audiotaping to help us summarize the discussion later, but no one will be given tapes outside of MPR
To help ensure we have a clear tape, we ask that only one person talk at a time
My colleague may also be taking some notes during the discussion
Time restrictions—moderator must move discussion to all topics
Offer opinion even if different from others; no right or wrong answers
B. PARTICIPANT INTRODUCTIONS (10 MIN)
Please introduce yourself by telling us:
Your first name
When and where you attended the PREP workshop
C. REFERRAL TO AND ENROLLMENT IN PREP (20 MIN)
How did you learn about this workshop?
Did you seek out the workshop after seeing a flyer, poster, or brochure, visiting a website, or hearing about it on a public service announcement or at a community event?
Did you hear about it from a friend, neighbor, or acquaintance?
Did a representative of any organization, such as a health clinic, a church, or the workshop leader recommend the workshop to you? If so, who recommended it?
If you first heard about it from an organization representative, what did that person tell you about the workshop before you enrolled?
What were your reasons for participating in the program?
What interested you about the workshop? How did you think it would benefit you?
What concerns did you have, if any? How were those concerns addressed?
Were you eligible for any special benefits, like gift cards or door prizes, by participating in the program?
Did you receive any other benefits or services because of your participation in the workshop?
What effect did these benefits have on your motivation to join or continue participating in the PREP workshop?
D. EXPERIENCES with PREP workshop (20 MIN)
Describe a typical PREP class/meeting for me.
How many other people were in your group? How often did you meet, for how long at a time, and at what time of day? Where were the meetings held?
Did the program provide childcare during meetings? What about transportation assistance? Were refreshments provided?
What kind of activities did you do during a typical workshop meeting? Did you listen to lectures? Have group discussions? Do written exercises? Watch videos?
Which activities did you like best?
Which did you like least?
What topics did you discuss during your sessions?
Which topics have been most useful for you?
Least useful?
Did any of your classes or sessions address issues of domestic violence, or dating violence, or other types of violence in relationships?
If so, how was it addressed?
How effective was your workshop instructor?
Was he/she able to convey the information clearly and in a way that you could readily understand?
Was the material presented in a way that kept your interest and attention? Or were you often bored?
Were examples and illustrations relevant to your situation and life?
In what areas might the instructor improve? In what areas did he/she excel?
How difficult or easy was it to understand and apply the information and skills covered in the class?
Did you attend by yourself, with a partner, or with someone else?
How many of the workshop/class meetings did you attend?
If you did not attend all of the meetings, please explain why you had to miss some of them.
If you dropped out of the program after some period, please explain when and why you dropped out.
E. participation and reactions (20 min)
What is the main thing you learned from this workshop/class?
Have you used any of the skills or lessons in your daily life? If so, which skills?
What is the current attitude toward marriage and relationships among your friends, or in your neighborhood or community?
Have your attitudes or thoughts about relationships or marriage changed as a result of this workshop/class? If so, what were these changes?
Do you think the workshop has helped you personally?
If so, how has it helped? How important would you say the workshop/classes have been in your life?
Have the workshop/classes improved your relationship with a significant other or with other family members? If so, how have they improved it?
What do you think works well in this workshop?
What changes would you suggest to improve the workshop?
Would you recommend attending a PREP workshop to your friends or family members? Why or why not?
What are the main attitudes about marriage in your community?
Do you think this type of workshop can changes people’s attitudes about marriage? In what ways?
F. WRAP UP AND POST-GROUP INFORMATION FORM (5 MIN)
Does anyone have any other thoughts or comments about your experiences in the PREP program?
COLLECT POST-GROUP INFORMATION FORM.
PAY PARTICIPANTS AS THEY LEAVE (SIGN RECEIPT LOG)
APPENDIX g
PARTICIPANT INFORMATION FORMS
Information Form for Workshop Participants
Information Form for Public Agency Workshop Leaders
Information Form for Community-Volunteer Workshop Leaders
APPENDIX h
FEDERAL REGISTER NOTICE
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Office of the Assistant Secretary for Planning and Evaluation
Proposed Information Collection Activity
Comment Request
Proposed Project
Title: Oklahoma Marriage Initiative Process Study.
OMB No.: New collection.
Description: The Oklahoma Marriage Initiative Process Study (OMIPS) is an opportunity to improve understanding of this initiative and suggest lessons from its implementation to inform similar state and federal efforts. The project will document and analyze the context from which the initiative developed; its overall goals and objectives; and its evolution, organizational structure, partnerships, and service delivery system. The Oklahoma Marriage Initiative (OMI), launched in 1999 and funded by the Oklahoma Department of Human Services (DHS), was designed to reduce divorce through a multi-sector strategy with broad community support and the delivery of marriage and relationship skills education. The process study will document the evolution of the OMI, its program strategy and design, the consequences of OMI implementation approaches, the role of research in shaping implementation strategy, and OMI goals for changing the systems and culture within which it operates. The study team will gather data through focus groups and semi-structured interviews, and from documentary information and administrative records. Semi-structured interviews will be conducted with lead program staff, including representatives from up to 11 public agencies who have participated in the initiative. Seven group interviews will be held with workshop leaders from public agencies; two additional group interviews will be held with workshop leaders in the community/volunteer sector. Seven focus groups will be conducted with participants who have taken workshops at public agencies; two additional focus groups will be held with participants in the community-volunteer sector.
Respondents: Respondents for the process study include lead representatives of public agencies in Oklahoma, workshop instructors at public agencies, workshop instructors who provide services in the general community, workshop participants at public agencies, and workshop participants in the community. Five data collection instruments will be used in the process study data collection. This includes a protocol for semi-structured interviews with lead staff of OMI programs at public agencies; two guides for group interviews with workshop leaders at public agencies and in the community; and two guides for focus groups with workshop participants at public agencies and in the community. One to four lead staff at each of nine public agencies are expected to participate in semi-structured interviews, for a total of 11 to 44 respondents. The attendance goal for each group interview and focus group is eight to 12 people in 18 groups, for a total of 144 to 216 respondents.
Annual Burden Estimates
Number of Average burden
Number of responses per hours per Total burden
Instrument Respondents respondent response hours
Lead Staff Public Agencies protocol 44 1 1.50 66
Int. Guide for WS Leaders at Public Agencies 84 1 1.50 126
Int. Guide for Community WS Leaders 24 1 1.50 36
FG Guide for WS Participants at Agencies 84 1 1.50 126
FG Guide for Community WS Participants 24 1 1.50 36
Estimated Total Annual Burden Hours 390
In compliance with the requirements of section 3506(c)(2)(A) of the Paperwork Reduction Act of 1995, the Office of the Assistant Secretary for Planning and Evaluation is soliciting public comment on the specific aspects of the information collection described above. Copies of the proposed collection of information can be obtained and comments may be forwarded by writing to the Office of the Assistant Secretary for Planning and Evaluation, [INSERT ADDRESS] Washington, DC 20447, Attn: ASPE Reports Clearance Officer. All requests should be identified by the title of the information collection. The Department specifically requests comments on: (a) Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility; (b) the accuracy of the agency's estimate of the burden of the proposed collection of information; (c) the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. Consideration will be given to comments and suggestions submitted within 60 days of this publication.
Dated: December __, 2006.
[INSERT NAME]
Reports Clearance Officer
| File Type | application/msword |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |
© 2024 OMB.report | Privacy Policy