Supporting_Statement for 0308 RQI 2009 with Part B

Justification

1. Circumstances that Make the Collection of Information Necessary:

The Promoting Safe and Stable Families Amendments of 2001 (Pub. L. 107–133) amended Title IV–B of the Social Security Act (42 U.S.C. 629– 629e) to provide funding for nonprofit agencies that recruit, screen, train, and support mentors for children with an incarcerated parent or parents. The mentoring children of prisoners program (MCP) is administered by the Family and Youth Services Bureau (FYSB) of the Administration for Children and Families in the Department of Health and Human Services.

The legislation includes requirements for grantees to provide information that can be used to evaluate outcomes for participating children, including information necessary to demonstrate compliance with requirements established by the Secretary for the program.

The legislation also requires the Secretary to evaluate the programs and report to Congress, over and above requirements in the Performance Budget or Program Assessment and Rating Tool (PART). Mentoring research has found that strong relationships between mentors and children increase the likelihood of favorable outcomes such as academic progress and positive behavior.

Sections of legislation relevant to information collection

PUBLIC LAW 107–133—

Subtitle B—Mentoring Children of Prisoners

SEC. 439. GRANTS FOR PROGRAMS FOR MENTORING CHILDREN OF PRISONERS. 42 USC 629i.

[skip]

(d) APPLICATION REQUIREMENTS.—In order to be eligible for a grant under this section, the chief executive officer of the applicant must submit to the Secretary an application containing the following:

‘‘(1) PROGRAM DESIGN.—A description of the proposed program, including—

[skip]

(C) the number of mentor-child matches proposed to be established and maintained annually under the program;

(D) such information as the Secretary may require concerning the methods to be used to recruit, screen support, and oversee individuals participating as mentors, (which methods shall include criminal background checks on the individuals), and to evaluate outcomes for participating children, including information necessary to demonstrate compliance with requirements established by the Secretary for the program; and

(E) such other information as the Secretary may require.

2. Purpose and Use of the Information Collection:

Data will be analyzed and applied to program evaluation, management improvement, service quality, and grantee compliance with agreed upon goals. FYSB will use this information to assure effective service delivery and program management and to guide the development of national evaluation, monitoring and technical assistance systems.

Data from this collection will be used for reporting outcomes and efficiencies in the Performance Budget (under the Government Performance and Results Act/GPRA) and to meet commitments established during PART process, which the MCP program underwent during the FY 2005 assessment year. Data will also provide input for Congressional hearings and inform philanthropic interests and research efforts in addition to FYSB’s.

ACF’s aggregate caseload reporting system (see the following item 3) already collects caseload data at the mentoring agency level. That information will provide context and perspective for the (individual) relationship information to be gathered under this current proposal. FYSB is also initiating a national benchmarking evaluation of long-term child outcomes from mentoring. That study will measure child conduct, educational and developmental milestones, involvement in or avoidance of risk behavior, etc. FYSB views “relationship quality” as an intermediate outcome than can lead to favorable behavioral outcomes over a longer period of time. Outcome information will be integrated as it is obtained, leading to a continuous evaluation process. Outcome measurement features may also eventually be added to the online system for periodic reporting if these are demonstrated to be feasible.

3. Use of Improved Information Technology to Reduce Burden:

The instrument for the survey will be part of an online system developed by ACF (OMB 0970-0266) which has been in use since the beginning of FY 2005 for performance reporting and management oversight, The existing system collects quarterly caseload data, and grantees will be able to use a new module in the same system to complete interviews and submit results online, with privacy and security protocols in place. The data collected through the website is gathered and used for management and reporting purposes; no statistical analysis is used.

4. Efforts to Identify Duplication and Use of Similar Information:

MCP is an entirely new program targeted on a highly vulnerable population. While some grantees are experienced in operating mentoring programs, including those for children of prisoners, like Amachi sites, or general child mentoring, such as Big Brothers Big Sisters programs, many grantees are starting up for the first time. There is no existing system that collects the data called for or implied by the authorizing legislation.

Moreover, the data in this form is being collected from the same grantees by no other part of FYSB. Grantees routinely provide financial and narrative progress reports, and onsite monitoring protocols are under consideration, but the information in all these areas does not accomplish the objectives of the proposed collection. Nonetheless, useful connections can and will be made by linking quantitative, financial, narrative, and qualitative data as the program evolves.

5. Impact on Small Businesses or Other Small Entities:

FYSB’s approach to data collection and reporting is to minimize paperwork, eliminate unnecessary duplication, and allow service providers to spend most of their time providing services. In addition to the relationship instrument administered to the child, the survey will include general questions about the child such age, duration of mentoring relationship, frequency of contacts, etc. to establish context. These will be answered by the agency that administers the survey. The agency will also answer questions about organizational structure and other factors, but only once, not for each child. FYSB will not require the agency to interview every child in its caseload but only those in relationships at least nine months, which will be considerably fewer. This is to allow time for the nature of the relationship to develop and unfold. Mentoring outcomes take from six months to a year to be evident. FYSB will not require agencies to seek out children who are no longer in contact with the agency.

6. Consequences of Collecting the Information Less Frequently:

In its PART documents, FYSB committed to reporting relationship quality data as an annual performance measure. Annual measurement of relationship quality is less frequent than some researchers have used, whereby the interview is conducted with each child several times over a number of months for greater precision. This approach has its merits and may be considered in connection with the one-time outcome evaluation, for which a separate information collection application must be submitted.

FYSB will not require it on a more frequent than annual basis, since relationship factors have impacts on youth development that emerge gradually over time. The burden on programs would be excessive, and programs are already carefully monitoring mentoring pairs in consideration of the child’s well-being and best interest.

7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5:

None are applicable.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency:

The 60 day Federal Register notice was published January 8, 2009, Vol 74, page 814.

FYSB has engaged consultants and other experts in designing this data collection, including the scholar directly involved in the research upon which the instrument is based: a questionnaire developed specifically for mentoring programs which has been validated and tested extensively.¹

Dr. Jean Rhodes is an eminent scientific authority on mentoring programs, and she provided guidance and input during the design process, including features that are new to the instrument, such as questions about the agency. Guidance was also provided by the research firm of Abt Associates, Inc., of Cambridge Massachusetts, which has been working with FYSB on the outcome evaluation. In the past this organization has conducted evaluations of other mentoring programs for at risk youth that are similar to MCP.

During the initial creation of the instrument, FYSB submitted the instrument for comment by other partners in MCP, such as National Big Brothers Big Sisters of America and MENTOR/The National Mentoring Partnership.

Previous testing has shown that the questions are all sufficiently simple to be answered by children from age 5 up. Surveys from younger children can be carefully reviewed for evidence of error, and our instructions to grantees will remind them to be sensitive to age differences in explaining the survey and its purpose. To reduce the possibility of misunderstanding, FYSB will consider instructing that surveys be limited to children above 7 or 8. This will also reduce the collection burden somewhat. However, there were no comments suggesting young children would have trouble understanding.

9. Explanation of Any Payment or Gifts to Respondents:

There is no Federal remuneration of any kind for respondents. It is possible that individual agencies, at their own discretion, may sponsor recreational activities or treats to attract youth to their location and to enlist them in the mentoring program. Such events would not be likely to include participation in the survey, because of its confidential nature, but FSYB does not regulate the program at that level of detail. In any case, agencies may appropriately use their own (matching or other) funds in many ways. It should be noted that both youth and adult mentors participate in the general program on a voluntary, non-compensated basis.

10. Assurance of Confidentiality Provided to Respondents:

Protection of privacy and individual case files is a responsibility of the agency, which will be reminded of this. FYSB may examine the agency’s general diligence in this regard through onsite monitoring or other means. The agency must assure the child and consenting parent that the individual identity of the child will not be disclosed and only used securely within the agency as a management tool. The data forwarded to FYSB will contain only anonymous coding to connect survey responses with demographic information but not identifies. FYSB does not collect any data on the identities of the youth or volunteer adults in the MCP program. The online system is operated by ACF’s agency-wide contractor and contains numerous security and privacy protections, including passwords and user id’s.

11. Justification of Sensitive Questions:

The instrument assesses how satisfied the youth (mentee) is with the relationship; whether the mentee is happy in the relationship; whether the mentee trusts the mentor; and whether the mentor has helped the mentee to cope with problems, without specifying those problems. There are no topics raised in terms generally regarded by social science researchers as sensitive (e.g., there no questions regarding sexuality, health status, or financial status).

12. Estimates of Annualized Burden Hours and Costs:

Instrument	Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours
Relationship Quality Instrument for Mentoring Children of Prisoners Program	215 MCP grantees serving a total of approximately 25,000 children in the active annual caseload	116 (average caseload per MCP grantee)	1	24,940
Estimated Total Annual Burden Hours:				24,940

The actual burden will be considerably less since only children in relationships of more than nine months will be surveyed.

Task / Item	Annual Number	Annual Cost	Estimated Annual Cost
Training: To be provided by FYSB as a workshop at the national MCP conference with no charge to respondents. If form instructions are not well understood, grantees can contact a technical support hot line or call FYSB.	None	None	None
Hardware: Respondents already submit caseload reports (OMB 0970-0266) via the ACF Online Data Collection System, maintained by ACF at no cost to grantees. This report would be submitted through that system with no additional cost.	None	None	None
System Maintenance: Same as for Hardware.	None	None	None
Supplies (Diskettes, Mail, etc) Agencies may utilize existing, inexpensive storage media for their own record keeping purpose.	Not significant	Not significant	Not significant
Total for all grantees	Not significant	Not significant	Not significant

Estimated annual cost: 0

Estimated annual personnel cost @ $25/hr X 24,940 hours for recordkeeping, compilation and data: $623,500

Estimated total annual cost: $623,500

Notes: Agencies include these costs in grant budgets as part of their competitive proposals.

13. Estimate of Other Total Annual Cost Burden to Respondents and Record Keepers:

No costs

14. Annualized Cost to Federal Government:

Task / Item	Estimated Annual Cost
Contractor services to maintain online system	50,000
Federal Gov’t Staff (program analysis officer @ .15 FTE)	$17,700
Cost of instrument	No charge
Overhead	$5000
Total	$72,700

Estimated Annual Cost Burden for Federal Government: $72,700

15. Explanation for Program Changes or Adjustments:

There is no program change; however, there is an adjustment due to the method used in computing responses and respondent costs.

16. Plans for Tabulation and Publication and Project Time Schedule:

FYSB will compile the data and, as discussed earlier in this document, e.g., under “Use of Data” and “Consequences,” apply it to numerous objectives. A basic review and summary of the data will enhance the outcome evaluation discussed under several sections of this application. Publication of aggregate findings will take place via print or website display or distribution as documents via electronic means is a possibility. For example, national level data will be reported to OMB and Congress in the Performance Budget and PART updates, which are published after the President’s Budget is delivered to Congress.

FYSB submits 2nd Federal Register Notice for publication	May, FY 06
End of 2nd 60 day comment period	July, FY 06
Online systems development and grantee training	June-July, FY 06
FYSB distributes final version subject to approval	July, FY 06
Agencies administer surveys and collect data	August – September, FY 06
Results submitted to FYSB via online system	October 30, FY 07
FYSB and consultants analyze data and submit through performance budget	October-November, FY 07
Annual process thereafter	ongoing

17. Reason(s) Display of OMB Expiration Date is Inappropriate:

Not applicable

B. Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When item 16 is checked "Yes," the following documentation should be included in the supporting statement to the extent that it applies to the methods proposed:

1. Describe (including numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Respondents	Universe	Sample
MCP Participant Youth	25,000	6,000
MCP Grantees	215	n/a

This survey does not use a random sample. The children must meet two eligibility requirements to participate: They must be at least nine years of age and they must be in their mentoring relationship for nine months or longer. These requirements are part of the survey design.

2. Describe the procedures for the collection of information

Grantee staff are trained on how to collect this data, and they also have access to technical assistance during the annual data collection period. Grantees are to survey children over the telephone after obtaining caregiver permission.

. Statistical methodology for stratification and sample selection

Not applicable.

• Estimation procedure

Not applicable.

• Unusual problems requiring specialized sampling procedures

Not applicable

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Not applicable.

3. Describe methods to maximize response rates and to deal with issues of nonresponse. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

The federal program office provides routine reminders and ongoing assistance before and during the collection period. The training that is provided at the annual national conference on the survey is mandatory.

4. Describe any tests of procedures or methods to be undertaken.

Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for collection of identical information from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Not applicable

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Family and Youth Services Bureau: (202) 205-5916 Julie Hocker