Promoting Cultural Competence and Diversity in Youth Mentoring Programs Toolkit
Caliber, an ICF International Company (Caliber), in partnership with MENTOR/National Mentoring Partnership (MENTOR) and national mentor expert Dr. Jean Rhodes is conducting an assessment of the mentoring community to identify and assess needs for the purpose of building a toolkit of practical information and tools to assist mentoring programs in promoting cultural competence and diversity in their programs for the Family Youth Services Bureau (FYSB) within the Administration of Children and Families (ACF). This toolkit builds on the findings from earlier research conducted by Caliber for FYSB, which explored the issues and challenges around same-race and cross-race matching faced by directors of mentoring programs across the country. The initial research recommended that a comprehensive needs assessment be conducted to inform the development of this toolkit. This needs assessment would build upon the literature review and interviews conducted in Phase I of this work, refining that work by extending it to a larger and more representative group of mentoring programs. Information will be collected from the Mentoring Children of Prisoners (MCP) program grantees and non-MCP national mentoring programs.
A 2003 report by the White House Task Force for Disadvantaged Youth found that the Federal government funds only three programs in which mentoring is the primary focus. One of which is the Department of Health and Human Services’ Mentoring Children of Prisoners program, administered under FYSB within ACF, authorized by the Promoting Safe and Stable Families Act of 2001 (SSFA, P.L. 107-133). The MCP program is designed to nurture children in the nation with one or both parents incarcerated. In FY 2003, MCP grants were awarded for a period of three years to qualifying programs. During its first year, 9.3 million dollars were appropriated to grantees, and increased to close to 50 million in subsequent years.
The HHS Secretary is mandated to appropriate funds for the MCP grant program, specifically for evaluation, research, training, and technical assistance. In FY 2004, grantees began submitting progress reports to HHS. Two major overall findings were that programs had problems recruiting and retaining mentors.
Based on these findings and the findings from the Phase I report, FYSB within the ACF, will conduct a needs assessment to inform the development of toolkit modules related to minority mentor recruitment, assessment and matching, training, educating program staff and participants, and promoting ethnic identity development, which is further detailed in section 2.
The Phase I evaluation report recommended the development of a toolkit for program administrators that addresses issues related to racial matching in youth mentoring. FYSB will use the information collected to assess the needs of its Mentoring Children of Prisoners (MCP) grant program and other national mentoring programs and develop toolkit modules related to minority mentor recruitment, assessment and matching, training, educating program staff and participants, and promoting ethnic identity development. This information will inform the entities who are developing the toolkit to assist mentoring programs in promoting cultural competence and diversity in their programs.
This needs assessment serves three purposes:
It will help to identify the specific contents for the toolkit and its modules by providing greater detail about the characteristics of the programs that would be using these tools (e.g., their program practices, populations served, and characteristics of their environment).
It will help to identify additional modules (by gathering information from a larger group of programs) of the toolkit that might not have been identified in the first phase of this work or to identify modules from the first phase that are a lower priority for program directors.
It will guide the forthcoming evaluation of the MCP grant program, as a subset of questions will be asked of the MCP grantees.
The Caliber/MENTOR team plans to not only develop an initial prototype of the toolkit but to also further the understanding of the technical assistance needs of mentoring programs in promoting cultural competence and diversity. The toolkit is targeted to all mentoring programs that serve youth using one-on-one (programs that only offer e-mentoring are excluded). There is a particular focus on Mentoring Children of Prisoners (MCP) grantees, given FYSB’s role in administering these grants. However, these materials would also be relevant to other programs that use different one-on-one mentoring approaches (e.g., group mentoring, team mentoring) and that target other populations (e.g., low-income children, children of single parents).
The resources for Phase II (toolkit) will be used to develop and implement the needs assessment and to develop a prototype for the first module of the toolkit. Given the level of interest in minority mentor recruitment, it was recommended that this first module concentrate on the recruitment of minority mentors, providing specific materials to aid programs in recruiting African American mentors, in particular.
The needs assessment will involve one type of data collection—Web-based surveys—using Survey Monkey, an online data collection program.
An online survey approach was selected based on several advantages that include:
Easing burden by eliminating the labor expended to complete and mail a hard-copy survey.
Widening accessibility to the survey by virtue of its electronic, Internet-based form.
Providing Web-based technical support via electronic mail.
We have determined that the Web-based survey would be more feasible since it allows for better accessibility among participants and encourages a higher response rate.
The evaluation team will provide all 100 pilot sites with a Web-based feedback survey to elicit information on the usefulness of the toolkit, as well as ways in which the prototype might be further strengthened. As previously mentioned, we have determined that the Web-based survey would be more feasible since it allows for better accessibility among participants and encourages a higher response rate.
4. Efforts to Identify Duplication
As mentioned in response to item 1, this is the first comprehensive needs assessment being done in the field of mentoring. The needs assessment was developed in coordination with the Mentoring Children of Prisoner’s Grant Program Technical Assistance provider to avoid duplication of effort and undue burden on the grantees. Additionally, the needs assessment was developed in coordination with current efforts underway to prepare for an evaluation of the MCP grant. Where possible, questions have been incorporated into the needs assessment that will provide useful information for the evaluation and avoid multiple surveys for the MCP grantees.
5. Impact of Data Collection on Small Entities
This data collection effort will impact MCP grantees and other non-MCP programs. Efforts to reduce burden have been described in response to item 4.
If this data collection is not conducted, FYSB will not be able to assess current program practices related to recruitment of minority mentors and assignment of mentors to mentees or obtain suggestions that provide guidance on the development of the toolkit for program improvement. This is a one time project.
There are no special circumstances that would cause data collection to be conducted in a manner as follows:
Requiring respondents to report information to the agency more often than quarterly;
Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
Requiring respondents to submit more than an original and two copies of any document;
Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;
In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
Requiring the use of statistical data classification that has not been reviewed and approved by OMB; or
That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impeded sharing of data with other agencies for compatible confidential use; or
Requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that is has instituted procedures to protect the information’s confidentiality to the extent permitted by law.
A notice of public comment was issued by FYSB to elicit comments from interested parties outside of the agency. The first noticed was issued in the Federal Register/Vol. 71, No. 96/Thursday, May 18, 2006/Notices, pages 28870-28871. The second notice was issued in the Federal Register/Vol. 72, No. 35/Thursday, February 22, 2007/ Notices, pages 7998-7999. No comments were received as a result of the Federal Register notices, however, comments were received as a result of a pilot test of the needs assessment.
The Web-based needs assessment survey was pilot tested by nine participants from mentoring programs across the country November through December, 2006. In addition, six of these participants were involved in a workshop conducted at the Mentoring Children of Prisoners Grantee Meeting in November 2006. All participants were asked to provide feedback on the survey and to offer recommendations for toolkit development. This feedback was taken into consideration as the instrument was revised.
Overall, respondents found the survey easy to use. Respondents indicated that the length of the survey was very reasonable considering how much information was covered. Respondents liked the user-friendly format of Survey Monkey, the Web-based survey system used. Respondents found the types of questions probed most appropriate for youth mentoring programs. Some program sites do not have reliable computers so it was recommended that the toolkit include hard copies of manuals and other materials. The importance of keeping the toolkit cost effective was emphasized since many mentoring programs are limited in their discretionary funds.
Non-MCP grantees that participated in the Web-based needs assessment pilot test received $25 gift cards, and MCP grantees that participated in the workshop discussion were provided lunch. All participating programs that complete the Web-based needs assessment will be entered into a drawing to win an Apple iPod nano. We will be conducting pilot tests toolkit prototype with volunteers from 50 MCP and 50 non-MCP programs who participated in the needs assessment. Pilot program participants will receive a CD version of the final toolkit prototype when it becomes available.
Caliber, an ICF International Company, established an Institutional Review Board (IRB) in 2002. The official date of approval of our application for a federal-wide assurance was April 17, 2002 with an expiration date of April 6, 2008. The expiration date for our IRB is July 12, 2008. Our IRB # is 00001920 and the federal-wide assurance # is 0002349. Caliber holds monthly meetings of its members and reviews IRB packages prepared by project managers during those meetings. Data collection is authorized to proceed only upon approval of a quorum of the membership or the review by a member designated by the IRB Chair for an expedited review. We have attached a form that all Caliber project managers complete at the time of submission of a proposal, indicating whether IRB review will be required.
We will provide survey respondents with a statement of confidentiality, which they must agree to prior to taking the survey. This agreement presents the purpose of the evaluation, the importance of participant input, and a voluntary participation statement.
We will not be asking questions that are of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private for this data collection effort.
We began this information collection with the following burden estimates, which were posted in the Federal Register notices.
ANNUAL BURDEN ESTIMATES |
||||
Instrument |
Number of respondents |
Number of responses per respondent |
Average burden hours per response |
Total burden hours |
Mentoring ToolKit Web-based Needs Assessment Questionnaire.................................................................. |
442 |
1 |
.75 |
332 |
Mentoring ToolKit Web-based Focus Group.............................................................................. |
40 |
1 |
1 |
40 |
Mentoring ToolKit Web-based Feedback Questionnaire………….……………………………….
Estimated Total Annual Burden Hours................. |
100 |
1 |
.25 |
25 |
.................... |
.................... |
........................ |
397 |
|
There was a reduction in burden from the 60-day to the 30-day notice at the agency’s discretion. There was a reduction in the number of Mentoring Children of Prisoners (MCP) grantees who will be participating in the Web-based needs assessment questionnaire from 221 to 157. The sample of 221 non-MCP grantees remained the same. This brings the total number of survey participants from the original number of 442 (221+221) to the current number of 378 (157+221). There was also a reduction in burden because the Web-based focus groups to be conducted with 40 program administrators, was eliminated.
REVISED ANNUAL BURDEN ESTIMATES |
||||
Instrument |
Number of respondents |
Number of responses per respondent |
Average burden hours per response |
Total burden hours |
Mentoring ToolKit Web-based Needs Assessment Questionnaire.................................................................. |
378 |
1 |
.75 |
283.5 |
Mentoring ToolKit Web-based Feedback Questionnaire………….……………………………….
Estimated Total Annual Burden Hours................. |
100 |
1 |
.25 |
25 |
.................... |
.................... |
........................ |
308.5 |
|
A pre-test involving mentoring programs that are not involved with the evaluation project estimates that it will take approximately 45 minutes to complete the Web-based needs assessment questionnaire and 15 minutes to complete the Web-based feedback questionnaire. We expect to administer the one time Web-based needs assessment questionnaire to 378 program administrators. Program administrators will spend 283.5 hours completing this questionnaire. We expect to administer the one time Web-based feedback questionnaire to 100 program administrators. Program administrators will expend 25 hours completing this questionnaire. Below is the new annul burden estimate based on the changes previously described.
An estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information is $5,5651 based on the new burden estimates presented previously. This amount represents a one-time cost. It relates directly to the hours expended by program administrators as respondents. This is not an annual data collection activity.
The Family and Youth Services Bureau within the Administration of Children and Families and the contractor estimate the data collection and analysis for this survey will come to approximately $37,000 for data collection, analysis and reporting.
15. Explanation for Program Changes
There are no program changes or adjustments reported in this Supporting Statement because this is a new project.
16. Plans for Tabulation and Publication
The data collected from the comprehensive needs assessment will be analyzed and used to inform the development of the toolkit prototype. We will use Survey Monkey’s analysis tools to analyze the data submitted by survey participants. We will be looking at frequencies and conducting cross-tabulations of the data.
17. Approval to Not Display the Expiration Date for OMB Approval
There is no request to not display the expiration date for OMB approval.
18. Exception to the Certification Statement
There is no request for an exception to the certification statement as outlined in Form OMB 83-I.
The evaluation team will administer the online needs assessment questionnaire to program administrators at a sample of 157 MCP grantees and 221 non-MCP programs. These numbers represent the potential universe of individuals eligible to respond to the Web-based questionnaires. Our procedures for stratification and sample selection are discussed below.
The following describes the procedures for the collection of information.
Statistical Methodology for Stratification and Sample Selection
The sampling strategy for the survey will be stratified into two segments: MCP grantees and other face-to-face mentoring programs. To ensure that the data collected from MCP grantees serves as many uses as possible, we propose to administer the survey to 157 MCP grantees. In addition to the sample of MCP grantees, we will draw a sample of other one-on-one mentoring programs from among MENTOR’s list of over 4,000 mentoring programs. Based on this list, we will conduct a stratified random sample using key characteristics such as: (1) the race/ethnic makeup of each program, (2) the location of each program [e.g., urban/rural], and (3) the size of each program, to select program administrators from 221 programs, thus, ensuring representation of MCP grantees and other programs.
To ensure that the materials developed for the prototype are useful to those in the field, we will conduct pilot tests with 50 MCP grantees and 50 non-MCP programs. Programs participating in the needs assessment will be invited to participate in the pilot. To encourage participation, programs will be provided with a CD version of the final prototype upon completion of the project. If more than 50 MCP grantees and 50 non-MCP programs volunteer for the pilot, we will randomly select from among the volunteers. It is necessary to limit the pilot to 100 total programs in order to complete the pilot within the designated timeline. The piloting process will include gathering feedback on the prototype through feedback forms. The toolkit development team will provide all 100 pilot sites with a link to the Web-based feedback form to elicit information on the usefulness of the toolkit, as well as ways in which the prototype might be further strengthened.
We will test for bias by comparing key characteristics of the 221 non-MCP programs in the sample with MENTOR’s overall list of 4,000 programs. Such key characteristics may include: (1) the race/ethnic makeup of each program; (2) the location of each program [e.g., urban/rural]; and (3) the size of each program. For the pilot test, we will use volunteers to develop our sample of 100 participants. If significant bias is identified we will either weight the results (in the case of the web-based survey) or resample (in the case of the pilot study). Weighting of the comparison group will be performed using the same key characteristics identified in the test for sample bias.
Degree of accuracy needed for the purpose described in the justification There will be a margin of error within ten percent of the true population.
Unusual problems requiring specialized sampling procedures
We foresee no unusual problems that will require specialized sampling procedures.
Any use of periodic (less frequent than annual) data collection cycles to reduce burden
This is a one-time data collection effort with no plans for future data collection.
Our main method of survey administration is Web-based. This method was selected because the target populations are familiar with computer technology and past experience shows that having to physically fill out a questionnaire and mail it in takes more perceived effort on the part of the respondent. The Web-based method allows the respondents to complete the questionnaire and submit it with ease.
We further intend to make additional contact with respondents who have not completed and submitted surveys within two weeks of notification of instrument availability.
The Web-based needs assessment survey was pilot tested by nine participants from mentoring programs across the country November through December, 2006. In addition, six of these participants were involved in a workshop conducted at the Mentoring Children of Prisoners Grantee Meeting in November 2006. Results of the tests ensured that the information in the Web-based survey presented was clear and easy to follow. We also will be conducting a pilot test of the toolkit prototype before it is finalized with 50 MCP and 50 non-MCP grantees.
The evaluation team consists of Caliber, an ICF International Company (Contact: Dr. Heather Clawson, 703-385-3200), in partnership with MENTOR/National Mentoring Partnership (Contact: Dr. Tonya Wiley 703-224-2200) and national mentor expert Dr. Jean Rhodes (617-236-8060). Caliber will be primarily responsible all data collection efforts, data analysis and reporting.
Dr. Jean Rhodes, a nationally recognized expert and researcher in mentoring programs is a professor of Psychology at the University of Massachusetts in Boston. As the expert advisor on this project, she provided information on the statistical methodology for stratification and sample design and the estimation procedure.
| File Type | application/msword |
| File Title | OMB CLEARANCE PACKAGE |
| Author | robinsok |
| Last Modified By | Julie E. Hocker |
| File Modified | 2007-05-02 |
| File Created | 2007-05-02 |