Transformation Accountability (TRAC) Reporting System
Supporting Statement
The Substance Abuse and Mental Health Services Administration’s (SAMHSA), Center for Mental Health Services (CMHS) is requesting approval from the Office of Management and Budget (OMB) for a revision to the TRAC reporting system data collection (0930-0285, expiration 5/31/13) which include the following three instruments:
1. The CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services (Attachment 1);
2. The CMHS NOMs Child Client-level Measures for Discretionary Programs Providing Direct Services (Child/Caregiver Version) (Attachment 2)
3. The Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators (Attachment 3).
CMHS requests approval to:
Increase the number of questions in the instrument due to the agency’s need for
additional information from its programs to satisfy reporting needs.
Specifically, CMHS proposes to add 6 items in a new section entitled “Violence and Trauma” and 6 items in a new section entitled “Military Family and Deployment” to the CMHS NOMs instruments (items 1 and 2 above). The purpose of this data collection is to provide performance monitoring data in support of two of SAMHSA’s 8 Strategic Initiatives: Trauma and Justice and Military Families. It will add approximately nine minutes per survey to the current data collection activity. No additions are proposed to the Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators.
The six new items are collected using a services tool the provides CMHS the capacity to monitor performance for all of its discretionary program: particular populations served, numbers of people served, types and locations of particular activities supported, effectiveness across programs for particular populations, the characteristics and effectiveness across programs of activities relative to national, subpopulation and geographic area data and trends. In order to be fully accountable for the spending of federal funds, SAMHSA/CMHS requires all its services programs to collect and report data on all consumers served as a means of ensuring that program goals and objectives are being met. Data collected as part of this package are used a tool to monitor performance through the grant period and ensure appropriate spending of federal funds.
Approval of this information collection will allow SAMHSA to continue to meet Government Performance and Results Act of 1993 (GPRA) reporting requirements that quantify the effects and accomplishments of its discretionary grant programs which are consistent with OMB guidance.
In order to carry out section 1105(a) (29) of the GPRA, SAMHSA is required to prepare a performance plan for its major programs of activity. This plan must:
a) Establish performance goals to define the level of performance to be achieved by a program activity;
b) Express such goals in an objective, quantifiable, and measurable form;
c) Briefly describe the operational processes, skills and technology, and the human, capital, information, or other resources required to meet the performance goals;
d) Establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity;
e) Provide a basis for comparing actual program results with the established performance goals; and
f) Describe the means to be used to verify and validate measured values.
SAMHSA’s legislative mandate is to increase access to high quality prevention and treatment services and to improve outcomes. Its mission is to reduce the impact of substance use and mental illness on our communities.
All of SAMHSA’s programs and activities are geared toward the achievement of goals related to reducing the impact of substance use and mental health disorders. GPRA performance monitoring is a collaborative and cooperative aspect of this process.
SAMHSA is striving to coordinate the development of these goals with other ongoing performance measurement development activities. This information collection is needed to provide objective data to demonstrate SAMHSA’s monitoring and achievement of its mission and goals.
These proposed data activities are intended to promote the use of consistent performance measures among CMHS-funded grantees and contractors. These common measures are a result of extensive examination and recommendations, using consistent criteria, by panels of staff, experts, and grantees. Wherever feasible, the measures are consistent with or build upon previous data development efforts. These data collection activities will be organized to reflect and support the domains specified for SAMHSA’s NOMs for programs providing direct services, and the categories developed by CMHS to specify the infrastructure, prevention, and mental health promotion activities.
Individuals at three different levels will use the information: the SAMHSA Administrator and staff, the Center Directors and Project Officers, and grantees:
SAMHSA level—This information will be used to inform the administration on the performance of the programs funded through the Agency. Assessment of performance will be based on the new measures in line with the grant’s program goals as set by program leadership. The intent is that the information will serve as the basis of the annual performance report to Congress contained in the Justifications of Budget Estimates.
Center level—In addition to providing information on the performance of the various programs, the information can be used to monitor and manage individual grant projects within each program. The information can be used to identify strengths and weaknesses and provide an informed basis for providing technical assistance and other support to grantees, informing continuation funding decisions, and identifying potential subjects for further analysis.
Grantee level—In addition to monitoring performance outcomes, the grantee staff can use the information to improve the quality of services that are provided to consumers within their projects, to promote service system capacity and infrastructure development, to prevent negative impacts of mental health problems, and to promote mental wellness.
To fulfill GPRA requirements SAMHSA develops a report for each fiscal year that includes results of performance monitoring for the three preceding fiscal years. The additional information collected through this process will allow SAMHSA to report on the results of these performance outcomes as well as be consistent with the specific performance domains that SAMHSA is implementing to assess the accountability and performance of its discretionary and formula grant programs.
To facilitate SAMHSA-wide reporting, the agency has identified ten domains of particular interest for accountability and performance monitoring of client-level data for programs providing direct services. These domains are:
Access/Capacity
Functioning
Stability in Housing
Education and Employment
Crime and Criminal Justice
Perception of Care
Social Connectedness
Retention
Cost-Effectiveness
Evidence-Based Practices
As stated above, the SAMHSA CMHS programs that provide direct treatment to consumers, or Services programs, currently have an OMB-approved data collection in place. Consequently, this request for approval of the two Services instruments is for revisions to the existing data collection instruments. This data collection includes separate data collection forms that are parallel in design for use in interviewing adults and children (or their caregivers for children under the age of 11 years old); named the CMHS NOMs Adult Client-level Measures for Discretionary Programs Providing Direct Services and the Child Client-level Measures for Discretionary Programs Providing Direct Services, respectively. These SAMHSA TRAC data will be collected at baseline, at six month reassessments for as long as the consumer remains in treatment, and at discharge. The data collection encompasses eight of the ten SAMHSA NOMs domains.
Table 1. Data Collection for Client-level Measures
Domain |
Number of Questions: Adult |
Number of Questions: Child |
Access/Capacity |
4 |
4 |
Functioning |
28 |
26 |
Violence and Trauma |
6 |
0 |
Military Family Deployment |
6 |
6 |
Stability in Housing |
1 |
2 |
Education and Employment |
4 |
3 |
Crime and Criminal Justice |
1 |
1 |
Perception of Care |
15 |
14 |
Social Connectedness |
4 |
4 |
Retention |
5 |
5 |
Total Number |
75 |
65 |
To facilitate CMHS reporting of GPRA data for programs engaged in substantial infrastructure development, prevention, and mental health promotion activities, the agency has identified 14 categories of particular interest for accountability and performance monitoring. No changes are proposed for these categories. These categories are:
Policy Development
Workforce Development
Financing
Organizational Change
Partnerships/Collaborations
Accountability
Types/Targets of Practices
Awareness
Training
Knowledge/Attitudes/Beliefs
Screening
Outreach
Referral
Access
The following table summarizes the total number of indicators for each category that may or may not apply to each grant program:
Table 2. Data Collection for Infrastructure, Prevention, and Mental Health Promotion Indicators
Category |
Number of Indicators |
Policy Development |
2 |
Workforce Development |
5 |
Financing |
3 |
Organizational Change |
1 |
Partnerships/Collaborations |
2 |
Accountability |
6 |
Types/Targets of Practices |
4 |
Awareness |
1 |
Training |
1 |
Knowledge/Attitudes/Beliefs |
1 |
Screening |
1 |
Outreach |
2 |
Referral |
1 |
Access |
1 |
Total Number |
31 |
SAMHSA and CMHS intend to compare infrastructure, prevention, and mental health promotion targets set at baseline with data collected quarterly. These outcomes will be used as the indicator of performance.
Proposed Changes to Data Collection Tool
CMHS has increased the number of questions in the Services instrument to satisfy Agency reporting needs. The following paragraphs present a description of the changes made to the information collection. These questions will be contained in new sections in the NOMS tool.
Violence and Trauma —CMHS added 6 items in a new section entitled ”Violence and Trauma”
Experiences with Violence and Trauma–One of SAMHSA’s 8 Strategic Initiatives is trauma and violence. In order to capture this information, CMHS is adding six new questions to be asked of respondents. This information will help in SAMHSA’s overall goal of reducing the behavioral health impacts of violence and trauma by encouraging substance abuse treatment programs to focus on trauma-informed services.
Military Family and Deployment—CMHS added 6 new items in a new section entitled “Military Family and Deployment”
Veteran Family Status and Areas of Deployment – SAMHSA is also interested in collecting data on active duty and veteran military members. Collection of these data will allow CMHS to identify the number of veterans served, deployment status and location, and family veteran status in conjunction with the types of services they may receive. Identifying a client’s veteran status and deployment area allows CMHS and the grantees to monitor these clients and explore whether special services or programs are needed to treat them for substance abuse and other related issues. Identification of veteran status and other military family issues will also allow coordination between SAMHSA and other Federal agencies in order to provide a full range of services to veterans. CMHS will also be able to monitor their outcomes and activities per the NOMS.
Information technology will be used to reduce program respondent burden. The existing TRAC System is a web-based data entry and reporting system designed to support web-based data collection efforts for CMHS. The system will be updated to incorporate proposed changes to the client-level data collection and the infrastructure development, prevention, and mental health promotion performance indicators. 100% of responses are expected to be submitted electronically through the web-based system. The TRAC System also provides a data repository service that includes methods for receiving the data, data quality checks, storage, and data presentation in reports by individual performance indicator or grouped with other performance indicators. The TRAC system complies with the requirements of Section 508 of the Rehabilitation Act to permit accessibility to people with disabilities.
This web-based system is intended to allow for easy data entry and access to reports for grantees that are required to submit TRAC data to CMHS. Entering and accessing data and viewing reports will be limited to those individuals with a username and password. A user’s level of access to the data and reports will be defined based on his or her authority and responsibilities.
Electronic submission of the data promotes enhanced data quality. With built-in data quality checks and easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once the data are put into the web-based system, it will be available for access, review, and reporting by all those with access to the system from Center staff to the grantee staff.
This data collection is necessary in order to assess grantee performance, and standardize data collection. Although individual CMHS programs currently collect and report some type of performance data, CMHS does not currently have standard reporting for the proposed items. Instead, individual grant programs are independently collecting data on a variety of indicators using different types of measures. This data collection effort aims to reduce the collection of duplicative information in these other data collection efforts by adding the proposed elements to an existing system from which a single data collection effort can support multiple analytic efforts (e.g., GPRA reporting, ad hoc requests, etc). This data collection attempts to standardize the indicators and the measures.
A program-level review of current measures and methods of collection was conducted to identify duplication of these data collection efforts. With the goal of creating standardized indicators and methods for monitoring grantee performance across the Center, existing measures were considered for use where appropriate. However, modification of current measures was necessary in some cases to generalize across varied programs. Each of these data collection instruments was reviewed and approved by the Government Project Officers, Branch Chiefs, and CMHS senior leadership as meeting the performance monitoring and management needs of individual programs and the Center. Since many of the grantees engaged in infrastructure development, prevention, and mental health promotion activities already collect data for the proposed indicators, the creation of this system will provide them with a standardized method for reporting to CMHS.
Individual grantees vary from small entities through large provider organizations. Every effort has been made to reduce the number of data items collected from grantees to the least number required to accomplish the objectives of the effort and to meet performance and GPRA reporting requirements and therefore, there is no significant impact involving small entities in general. Based on the pilot test and input and feedback from CMHS Project Officers, however, we understand that it may be difficult for some American Indian/Alaska Native Tribes and tribal organizations to report on the infrastructure development, prevention, and mental health promotion performance indicators on a quarterly basis. We will, therefore, develop a waiver process to allow such grantees to request, through their Project Officers, to report on these indicators every six months rather than quarterly.
Client-level data
Mental health programs typically collect client-level data at admission and then conduct periodic reassessments of consumers while the individual remains in services. When feasible, mental health providers also conduct an assessment when the consumer is discharged. The data collection schedule for the client-level measures parallels this model. All programs that provide direct services will collect data every six months while the consumer is receiving services; this is a reduction from the prior requirement of quarterly data collection for three of the CMHS programs (the National Child Traumatic Stress Initiative, Meeting the Needs of Elderly Americans, and HIV/AIDS Minority Mental Health Services programs.)
The baseline data collection point is critical for measuring changes. Extending the interval for the periodic reassessment beyond the requested intervals could lead to loss of contact with consumers, significantly diminishing the response rates and lowering the value of the data for performance reporting use by losing measurement of intermediate effects.
Infrastructure development, prevention, and mental health promotion data
This quarterly data collection requirement for the infrastructure development, prevention, and mental health promotion performance indicators is necessary to provide CMHS with the information when needed for appropriate program monitoring and management, as well as for GPRA performance reporting.
This information collection fully complies with the guidelines in 5 CFR 1320.5(d) (2).
The notice required in 5 CFR 1320.8(d) was published in the Federal Register on June 15, 2011 (76 FR 35000). No comments were received.
No monetary payment will be made to the mental health programs or to the consumers participating in data collection. No monetary payment is directly paid to grantees for the submission of the GPRA data. They are expected to provide this information as a requirement of their grant award.
The client-level measures require grantees to interview all consumers that they serve.
For the client-level data collection process, program respondents will be expected to meet the requirements of the HIPAA and its associated Privacy Rule that sets the standards for the use and disclosure of an individual’s health/mental health information. Since the data reported for each consumer will be provided to the CMHS contractor only by number and not by name, the data cannot be directly linked to a specific person. The grantee providing the data will maintain the link between the identifier and the name of the consumer. The CMHS contractor will not have access to existing consumer clinical records, which are under the control of the respondent programs. Neither the CMHS contractor nor CMHS can link individual consumers to the data reported by the respondent programs.
The infrastructure development, prevention, and mental health promotion data collection processes do not involve gathering client-level information. Program respondents will be expected to meet the requirements of the HIPAA and its associated Privacy Rule that sets the standards for the use and disclosure of an individual’s health/mental health information.
This project was given an exemption by Westat’s Internal Review Board (IRB) because it is considered performance reporting and no individual consumer identifiers are collected or submitted to Westat. However, grant projects use informed consent forms as required and as viewed appropriate by their individual organizations. They use the appropriate forms for minor/adolescent participants requiring parental approval.
The informed consent forms usually contain the following elements:
Explanation of the purpose of the program or research.
Expected duration of the subject’s participation.
Description of the procedures to be followed.
Identification of any procedures which are experimental.
Description of any reasonably foreseeable risks or discomforts to the subject.
Disclosure of appropriate alternative procedures or courses of treatment.
Statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained.
Contact names & phone numbers for participants to ask questions about program, participant rights, and injury.
SAMHSA’s mission is to improve the quality and availability of prevention, early intervention, treatment, and rehabilitation services for substance abuse and mental illnesses, including co-occurring disorders, in order to improve health and reduce illness, death, disability, and cost to society. In carrying out this mission it is necessary for grantees providing direct services to collect sensitive items such as criminal justice involvement, substance use, and data related to mental health functioning. The data that will be submitted by each grantee will be based in large part on data that most of the programs are already routinely collecting. This primarily includes data on consumer demographics, mental health condition/illness and treatment history, services received, and consumer outcomes. These issues are essential to the service context. Many grant projects use informed consent forms as required and as viewed appropriate by their individual organizations. They use the appropriate forms for minor/adolescent participants requiring parental approval.
The time to complete the revised instruments is estimated in Table 3. These estimates are based on grantee reports of the amount of time required to complete the currently approved instruments accounting for the additional time required to complete the new questions, as based on an informal pilot and prior CMHS experience in collecting similar data.
Table 3. Estimates of Annualized Hour Burden
Type of Response |
Number of Respondents |
Responses per Respondent |
Total Responses |
Hours per Response |
Total Hour Burden |
Hourly Wage Cost |
Total Hour Cost |
Client-level baseline interview |
15,681 |
1 |
15,681 |
0.481 |
7,527 |
$152 |
$112,903 |
Client-level 6-month reassessment interview |
10,637 |
1 |
10,637 |
0.367 |
3,904 |
$15 |
$58,557 |
Client-level discharge interview3 |
4,508 |
1 |
4,508 |
0.367 |
1,776 |
$15 |
$26,644 |
Client-level baseline chart abstraction4 |
2,352 |
1 |
2,352 |
0.1 |
235 |
$15 |
$3,528 |
Client-level reassessment chart abstraction5 |
8,703 |
1 |
8,703 |
0.1 |
870 |
$15 |
$13,055 |
Client-level discharge chart abstraction6 |
8,241 |
1 |
8,241 |
0.1 |
824 |
$15 |
$12,362 |
Client-level Subtotal7 |
|
|
|
|
15,137 |
$15 |
$227,048 |
Infrastructure development, prevention, and mental health promotion quarterly record abstraction |
942 |
4 |
3,768 |
4 |
15,072 |
$358 |
$527,520 |
TOTAL |
16,623 |
|
|
|
29,298 |
|
$885,135
|
There will be no capital, start-up, operation, maintenance, nor purchase costs incurred by the mental health programs participating in this CMHS data collection, or by consumers receiving CMHS-funded services.
The total contract award to cover all aspects of the design of the study, sampling design, data collection, and development of the data files, data tapes, and technical documentation is $8,807,858 over a 36-month period. Thus, the annualized contract cost is $2,935,953.
Additional costs will be incurred indirectly by the government in personnel costs of staff involved in oversight of data collection. It is estimated that one CMHS employee will each be involved for 100 percent of their time. Costs of CMHS staff time will approximate $120,000 annually.
The estimated annualized total cost to the government will be $2,989,494.
Currently there are 26,993 burden hours in the OMB inventory. CMHS is now requesting 29,298 hours. This increase of 2,305 hours is due to the additional questions.
Data for the annual performance plan/report are needed by SAMHSA on an ongoing basis. Data collection will commence with approval from OMB. Data are provided by CMHS for the most recently completed calendar year to SAMHSA each May in order to assure analysis in time for the annual performance report. The annual performance report must be submitted to the Department of Health and Human Services and to OMB by September and is included in the President's Annual Budget Request, which is released to the public February 1st. Data may be refined and added to the final Presidential Budget Request after the Department submits its initial performance report.
Data will be available to CMHS staff and grantees through a series of reports available through the web-based TRAC system. Assigned roles determine user access. Individual grantees will only be allowed detailed access to data from their grant. They will also have access to summary information across all grantees in their program. CMHS staff access will be determined by their span of responsibility.
The TRAC System includes web-based reports of the current client-level data including information on the number of consumers served, their demographic characteristics, baseline status, and change scores for the various outcome domains. These data and the additional items will be analyzed and presented in performance reports using basic descriptive statistics. On the principle outcome items (i.e., the 8 NOMs domains covered), a comparison of consumer status after receiving services with baseline data will be used to assess any change in status; users will also be able to compare any of the interviews completed by a consumer. The web-based reports will also allow users to create basic cross tabulations of the data.
Web-based reports will be built for the infrastructure development, prevention, and mental health promotion data collection efforts incorporating information related to the categories and indicators described above.
Data will be used to report to Congress regarding the CMHS’ performance as specified in the SAMHSA Annual Justifications of Budget Estimates. This will also allow CMHS staff to examine performance longitudinally, by program, or individual grantee.
In addition to the reports on the TRAC website, data will be utilized for specialized analyses as needs emerge. Individual grantees will be able to download their own data into an Excel spreadsheet for further manipulation or to transfer to a statistical package.
The expectation is that over time the results will be examined for subpopulations of interest within individual activities or in response to emerging policy issues. With these analyses the data would be exported to a statistical package such as SAS for more elaborate analytic work.
The expiration date for OMB approval will be displayed on all data collection instruments.
This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.
The data collection tool is administered to all consumers receiving services covered by grant funds. The tool is administered to all consumers at intake and all consumers are targeted for follow-up data collection. In order to ensure accountability for the spending of federal funds, CMHS has employed the use of these data as a performance management tool to ensure that grantees are meeting the goals and objectives of the program. Data are used to monitor performance throughout the grant period. The Public Health Service Act Sec. 501 [290aa] (d) (13) with respect to grant Programs authorized under this title, assure that-all grants that are awarded for the provision of services are subject to performance and outcome data collections. SAMHSA has operationalized these requirements to indicate the need for data to be collected on all clients served.
The table below indicates the number of grant programs (with the number of active grantees in FY 2010) for each of the TRAC data collection efforts:
Table 4. Data Collection Effort by CMHS-funded Program and Number of Active Grantees in FY 2010
CMHS-funded Program |
Total Number of Grants |
Client-level |
Infrastructure Development, Prevention, and Mental Health Promotion |
Child and Adolescent Mental Health and Substance Abuse State Infrastructure Grants |
7 |
No |
Yes |
Circles of Care |
8 |
No |
Yes |
Comprehensive Community Mental Health Services for Children and their Families Program |
65 |
Yes |
Yes |
Earmarks |
37 |
Yes |
Yes |
Garrett Lee Smith Campus Suicide Prevention Grant Program |
40 |
No |
Yes |
Garrett Lee Smith State/Tribal Suicide Prevention Grant Program |
49 |
No |
Yes |
Healthy Transitions Initiative |
7 |
Yes |
Yes |
Historically Black Colleges & Universities National Resource Center |
1, with 20 sub-grantees |
No |
Yes |
Jail Diversion |
14 |
Yes |
Yes |
Linking Actions for Unmet Needs in Children’s Mental Health |
18 |
No |
Yes |
National Suicide Prevention Lifeline |
1 |
No |
Yes |
Native Aspirations |
1 |
No |
Yes |
NCTSI Treatment & Service Centers |
13 |
Yes |
Yes |
NCTSI Community Treatment Centers |
44 |
Yes |
Yes |
NCTSI National Coordinating Center |
1 |
No |
Yes |
Mental Health Transformation State Incentive Grant |
9 |
Yes |
Yes |
Minority AIDS/HIV Services Collaborative Program |
16 |
Yes |
No |
Minority Fellowship Program |
5 |
No |
Yes |
Older Adults Targeted Capacity Expansion |
10 |
Yes |
No |
Primary and Behavioral Health Care Integration |
11 |
Yes |
Yes |
Safe Schools/Health Students Initiative |
175 |
No |
Yes |
Services in Supportive Housing |
55 |
Yes |
No |
State Mental Health Data Infrastructure Grants for Quality Improvement |
54 |
No |
Yes |
Statewide Consumer Network Grants |
31 |
No |
Yes |
Statewide Family Networks Grants |
48 |
No |
Yes |
Suicide Lifeline Crisis Center FUP Grants |
6 |
No |
Yes |
Total |
726 |
281 |
664 |
Information data collection procedures will be the responsibility of individual grantees and may vary by type of program.
Some grantees have service providers conduct client-level baseline and follow-up assessments, while others have grant evaluators perform this function.
Some grantees may wish to collect client-level information using paper and pencil methods. CMHS will provide downloadable paper versions of the data collection instruments to facilitate this process. These grantees will then submit their data electronically via a web-based data entry process. The data for those consumers with both baseline and periodic reassessment data are matched using a unique encrypted consumer identifier developed by the grantee. Grantees will be clearly instructed not to use identifying information (i.e., social security number or initials) as the consumer identifier.
Required data collection points are:
BASELINE: For consumers who have not previously been seen by the grantee, baseline data will be collected at admission. For consumers already enrolled in the program and continuing to receive services, administrative data should be submitted by the grantee within 30 days of initiating TRAC data collection. The timing of any subsequent data collection point(s) will be anchored to the baseline point the grantee indicates in this administrative record.
REASSESSMENT: CMHS requires client-level data collection every six months while the consumer is receiving CMHS-funded services. Ongoing periodic status review is viewed as consistent with good clinical practice.
DISCHARGE: Grantees must provide information on the type of discharge on all consumers who are discharged. When the discharge is a planned event, the consumer will also be asked the questions on the CMHS client-level data collection instrument. The one exception to this requirement is when a consumer had responded to these same questions within the past 30 days as part of a Reassessment.
Infrastructure development, prevention, and mental health promotion performance data are to be submitted quarterly by the grantee Project Directors through a web-based data entry system. Some programs may opt to keep track of their information using paper and pencil methods but are required to submit the data electronically within 30 days of the end of each quarter.
Each Services grantee collecting client-level data will have established its own procedures to collect baseline, periodic reassessment, and discharge data as part of the original protocol. For newly admitted consumers, baseline data collection typically occurs at the time of intake to the services program. All other data collection occurs as part of the normal course of service delivery, most likely by the primary provider assigned to the consumer. As noted, the timing of the periodic reassessment was chosen to coincide with normal clinical practice. Consumers are typically quite cooperative with grantee staff because of the relationship established during service provision. However, some consumers do not return to services during this timeframe. In these cases, the provider must complete an administrative discharge for the individual. Over the past two years, we have learned that many providers do not complete an administrative discharge in hope or anticipation that the consumer will return to services. As a result, the reassessment rate is impacted because the required data is not entered into TRAC within the reassessment window. New training procedures have been put in place to remind providers that an administrative discharge is needed if a discharge interview did not occur.
As a tool to increase the response rates when the consumer is still in services, the TRAC Reporting system automatically generates a report for grantees indicating when upcoming data submissions or interviews for existing consumer are due. This report is now active. Training on this report and other features of the TRAC Reporting system are provided to newly awarded grantees at national grantee meetings when possible. In addition to these training sessions, experts as well as selected grantees will be identified and asked to make presentations at national grantee meetings on the importance of quality and complete data collection, as well as TRAC system features to help facilitate consistency on consumer assessments at the appropriate intervals. Since these sessions are well attended by grantees, it is anticipated that these strategies will help to improve completion rates. The contractor also offers three annual refresher trainings via webinar to existing grantees to ensure the quality of the data collection and to help with grantee turnover.
Finally, CMHS has a Performance Review Board that meets once a year prior to grant continuation awards. In addition to other criteria, all grantees who have a reassessment rate below 50% appear on in TRAC Performance Report (TPR) listing all grantees who are “under-performing” in the TRAC system. This Report is shared with the Performance Review Board, and they work with the grantee’s project officer to improve their performance. Our goal is to raise the reassessment benchmark to 70% for the reassessment indicator in the TPR over the next few years.
All the data elements in the client-level data collection instruments were taken from established data collection instruments that have a history of use in the mental health field and have already been tested for validity and reliability, (i.e., the MHSIP, YSS-F, YSS, K-6, and ASSIST questions). In addition, for the domains that are not specific to mental health, CMHS has taken questions currently used by CSAT (OMB No. 0930-0208) that were drawn from widely used instruments and have been used for several years. These include three client-level domains (Employment/Education, Crime and Criminal Justice, and Stability in Housing) and one system-level domain (Access/Capacity), which depends on common demographics collected on consumers. The content of these questions was appropriate for use, but additional value options were defined to reflect issues specific to the populations served by CMHS. The benefits of using these measures include a history of use in monitoring the performance of CSAT grantees, the ability to conduct cross-Center comparisons, and use of measures previously approved by OMB.
The infrastructure, prevention, and mental health promotion data elements are drawn from these grant’s existing performance indicators and modified to allow consistent reporting for CMHS. A pilot of nine grant Project Directors was conducted using the attached instrument; results indicated these data are already part of routine data collection for most of the pilot participants or are consistent with their funded activities.
CMHS has contracted with Westat to provide support for the development and ongoing operational support for these data collection efforts, including statistical and analytic issues and the development of a web-based reporting the system. The Westat Project Director for this effort is: Jessica Taylor, Ph.D., (phone: 240-314-5852).
Danyelle Mannix, Ph.D. (phone: 240-276-1879) will serve as the SAMHSA Project Officer responsible for receiving and approving contract deliverables.
Attachment 1—Adult Client-level Measures
Attachment 2—Child Client-level Measures
Attachment 3—Infrastructure Development, Prevention, and Mental Health Promotion Performance Indicators
1 An increase of .147 hours of response (or ~9 minutes) for the Military Family and Deployment and Violence and Trauma questions.
2 Based on minimum wage.
3 Based on an estimate that 35 percent will leave the program annually, and it will be possible to conduct discharge interviews on 40 percent of those who leave the program.
4 Based on 13 percent non-response for those eligible at baseline (18,033); baselines are required for all consumers served or an admin baseline for non-responders.
5 Based on 40 percent non-response for those eligible for six-month reassessment.
6 Based on 60 percent non-response for those dischared.
7 This is the maximum burden if all consumers complete the baseline and periodic reassessment interviews.
8 To be completed by grantee Project Directors, hence the higher hourly wage.
| File Type | application/msword |
| File Title | Transformation Accountability (TRAC) Reporting System |
| Author | Jessica Taylor |
| Last Modified By | ctac |
| File Modified | 2012-02-09 |
| File Created | 2012-02-09 |