Recovery-SS-A-3.4.14

SAMHSA RECOVERY MEASUREMENT PILOT STUDY

SUPPORTING STATEMENT

1. JUSTIFICATION

A1. Circumstances Making the Collection of Information Necessary

The Substance Abuse and Mental Health Services Administration (SAMHSA), an operating division within the U.S. Department of Health and Human Services (HHS), is charged with reducing the impact of substance abuse and mental illness on America’s communities. In an effort to meet its mission, SAMHSA has identified eight Strategic Initiatives to focus its limited resources on areas of urgency and opportunity: (1) Prevention of Substance Abuse and Mental Illness; (2) Trauma and Justice; (3) Military Families; (4) Recovery Support; (5) Health Reform; (6) Health Information Technology; (7) Data, Outcomes and Quality; and (8) Public Awareness and Support. These Initiatives will enable SAMHSA to respond to national, State, Territorial, Tribal and local trends and support implementation of the Affordable Care Act and the Mental Health Parity and Addictions Equity Act. The proposed pilot project serves to support Strategic Initiative #4, Recovery Support. All of SAMHSA’s programs and activities are geared toward the achievement of goals related to reducing the impact of substance use and mental health disorders. Developing and refining a measure of recovery is a collaborative and cooperative aspect of this process. SAMHSA is striving to coordinate the development of these goals with other ongoing performance measurement development activities, for example, development of performance measures for reporting of activities. This information collection is needed to provide objective data to demonstrate SAMHSA’s monitoring and achievement of its mission and goals.

SAMHSA’s Center for Behavioral Health Statistics and Quality (CBHSQ) is proposing a pilot test of its Recovery Measure. As part of its strategic initiative to support recovery from mental health and substance use disorders, SAMHSA has been working to develop a standard measure of recovery that can be used as part of its grantee performance reporting activities.

Approval of these items by the Office of Management and Budget (OMB) will allow SAMHSA to further refine the Recovery Measure developed for this project. It will also help determine whether the Recovery Measure is added to SAMHSA’s set of required performance measurement tools designed to aid in tracking recovery among clients receiving services from the Agency’s funded programs.

This information collection will allow SAMHSA to continue to meet the Government Performance and Results Act (GPRA) of 1993 reporting requirements that quantify the effects and accomplishments of its programs, which are consistent with OMB guidance. In order to carry out section 1105(a) (29) of GPRA, SAMHSA is required to prepare a performance plan for its major programs of activity. This plan must:

• Establish performance goals to define the level of performance to be achieved by a program activity;

• Express such goals in an objective, quantifiable, and measurable form;

• Briefly describe the operational processes, skills and technology, and the human, capital, information, or other resources required to meet the performance goals;

• Establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity;

• Provide a basis for comparing actual program results with the established performance goals; and

• Describe the means to be used to verify and validate measured values.

In addition, this data collection supports the GPRA Modernization Act of 2010 which requires overall organization management to improve agency performance and achieve the mission and goals of the agency through the use of strategic and performance planning, measurement, analysis, regular assessment of progress, and use of performance information to improve the results achieved. Specifically, this data collection will assist SAMHSA’s efforts to report on a consistent set of performance measures across its various grant programs. SAMHSA’s legislative mandate is to increase access to high quality substance abuse and mental health prevention and treatment services and to improve outcomes. Its mission is to improve the quality and availability of treatment and prevention services for substance abuse and mental illness.

Based on current funding and planned fiscal year 2014 notice of funding announcements

(NOFA), the following SAMHSA grantee programs will be selected to participate in this pilot study in fiscal years 2014 and 2015: Behavioral Health Treatment Court Collaborative (BHTCC); Cooperative Agreements to Benefit Homeless Individuals (CABHI); and the Primary and Behavioral Health Care Integration (PBHCI). Data collected will be used by individuals at three different levels: the SAMHSA administrator and staff, the Center administrators and government project officers, and grantees.

A2. Purposes and Use of the Information Collection

The urgency of health reform compels SAMHSA to define recovery and to promote the availability, quality, and financing of vital services and supports that facilitate recovery for individuals. In addition, the integration mandate in title II of the Americans with Disabilities Act and the Supreme Court’s decision in Olmstead v. L.C., 527 U.S. 581 (1999) provide legal requirements that are consistent with SAMHSA’s mission to promote a high-quality and satisfying life in the community for all Americans.

In support of strategic initiative #4, SAMHSA’s first charge was to develop a working-definition of recovery. SAMHSA’s definition of recovery was arrived at primarily via stakeholder input, a series of facilitated discussions and a comprehensive review of the literature. SAMHSA defines recovery as “A process of change through which individuals improve their health and wellness, live a self-directed life, and strive to reach their full potential.”

After developing the recovery definition, and supporting guiding principles, SAMHSA leadership sought a means for operationalizing recovery among the SAMHSA grantee population.

In consultation with SAMHSA’s Center for Mental Health Services’ Recovery Measure Expert Panel, a group comprised of content experts from the recovery field as well as consumers in recovery themselves, the expert team identified a need to develop an instrument that can be used to assess the degree to which recovery is occurring. Staff from SAMHSA’s Center for Behavioral Health Statistics and Quality (CBHSQ) was then charged with identifying an appropriate recovery instrument to capture the four dimensions of recovery (i.e. home, health, purpose and community) that contribute to SAMHSA’s working definition of recovery. Based on an extensive literature review, SAMHSA identified the World Health Organization’s (WHO) Quality of Life (QOL)-8 as the existing tool best suited to capture these four dimensions. However, additional recovery-related questions are of critical importance to measuring recovery within grantee populations focused on substance abuse and mental health recovery. To this end, SAMHSA incorporated twelve of its existing set of Government Performance and Results Act (GPRA) measures into the proposed recovery tool. The proposed Recovery Measurement Pilot Project has been reviewed and approved by the Catholic University Institutional Review Board (FWA00004459). A copy of the approval certificate is provided as Attachment 1.

This project will assess the usability and psychometric properties of the proposed tool among a voluntary group of 2-3 SAMHSA grantees. SAMHSA has developed a short 20-item instrument that has been designed to capture all four of SAMHSA’s proposed dimensions of recovery – health, home, purpose, and community. This measure is comprised of questions from the World Health Organization’s Quality of Life tool (WHO QOL 8) and SAMHSA’s existing set of Government Performance and Results Act (GPRA) measures. Data will be collected at two time points – at client intake and at six-months post-intake. These are two points in time during which SAMHSA grantees routinely collect data on the individuals participating in their programs.

The WHO QOL 8 will assess the following domains using the items listed below:

Question Number	Item	Domain
1	How would you rate your quality of life?	Overall quality of life
2	How satisfied are you with your health?	Overall quality of life
3	Do you have enough energy for everyday life?	Physical health
4	How satisfied are you with your ability to perform your daily activities?	Physical health
5	How satisfied are you with yourself?	Psychological
6	How satisfied are you with your personal relationships?	Social relationships
7	Have you enough money to meet your needs?	Environment
8	How satisfied are you with the conditions of your living space?	Environment

In addition to the eight WHO QOL-8 items, the proposed Recovery Measure will include existing measures used as part of the GPRA (Attachment 2). These outcomes measures reflect SAMHSA’s desire for consistency in data collected within the Agency. SAMHSA has implemented specific performance domains called NOMs to assess the accountability and performance of its discretionary and formula grant programs. These domains represent SAMHSA’s focus on the factors that contribute to the success of mental health and substance abuse prevention and treatment. The Client/Participant Outcome Measures will address the following performance domains:

• Abstinence from Drug / Alcohol Use

• Employment / Education

• Social Connectedness

• Social Consequences from Drug / Alcohol Use

• Access / Capacity

The current performance measures that are contained in the tool include:

Question Number	Item
9	During the past 30 days, how many days have you used any alcohol?
10	Note: Ask this only if the number of reported drinks in Question #9 is greater than zero, and respondent is male: During the past 30 days, how many days have you had five or more drinks in a day? [CLARIFY IF NEEDED: A standard drink =12 oz. beer, 5 oz. wine, 1.5 oz. liquor. Note: As this only if the number of reported drinks in Question #9 is greater than zero, and the respondent is female: During the past 30 days, how many days have you had four or more drinks in a day? [CLARIFY IF NEEDED: A standard drink = 12 oz. beer, 5 oz. wine, 1.5 oz. liquor.
11	During the past 30 days, how many days did you use any illegal drugs, including prescription drugs that were taken for reasons or in doses other than prescribed?
12	The following questions ask about how you have been feeling during the past 30 days. For each question, please indicate how often you had this feeling. Nervous? Hopeless? Restless or fidgety? So depressed that nothing could cheer you up? That everything was an effort? Worthless
13	During the past 30 days, how much have you been bothered by these psychological or emotional problems?
14	I have family or friends that are supportive of my recovery.
15	In a crisis, I would have the support I need from family or friends.
16	I feel I belong in my community.
17	Please select the one answer that most closely matches your situation.  I feel capable of managing my health care needs:
18	Are you currently employed? [Note: CLARIFY BY FOCUSING ON STATUS DURING MOST OF THE PREVIOUS MONTH, DETERMINING WHETHER CONSUMER WORKED AT ALL OR HAD A REGULAR JOB BUT WAS OFF WORK.]
19	Are you enrolled in school or a job training program? [If enrolled, is that full-time or part-time?]
20	In the past 30 days, how many nights have you been homeless?

SAMHSA will use the WHO QOL-8 items and GPRA measures to report on the performance of its discretionary services grant programs. The performance measures information is used by individuals at three different levels: the SAMHSA administrator and staff, the Center administrators and government project officers, and grantees:

SAMHSA Level —The information is used to inform the administration of the performance of the programs funded through the Agency. The performance is based on the goals of the grant program and includes the NOMs. This information serves as the basis of the annual GPRA report to Congress contained in the Justifications of Budget Estimates.

Center Level—In addition to exploring the performance of the various programs, the information is used to monitor and manage individual grant projects within each program. The information informs the government project officers of the projects staff’s abilities to meet their individual goals. The information has been used by government project officers to make funding continuation decisions.

Grantee Level—In addition to monitoring performance outcomes, the grantee staff uses the information to improve the quality of treatment and prevention services that are provided to clients within their projects.

SAMHSA will use the data to aid in assessing the feasibility of adding a standard measure of recovery that may be used as part of its grantee performance reporting activities to aid in tracking recovery among clients receiving services from our funded programs. GPRA items included in the Recovery Measure will be used to meet annual reporting requirements and for annual reporting required by GPRA and for NOMs comparing baseline with discharge and follow-up data. GPRA requires that SAMHSA’s report for each fiscal year include actual results of performance monitoring for the three preceding fiscal years. The additional information collected through this process will allow SAMHSA to report on the results of these performance outcomes as well as be consistent with the specific performance domains that SAMHSA is implementing as the NOMs, to assess the accountability and performance of its discretionary and formula grant programs.

A3. Use of Improved Information Technology and Burden Reduction

In an effort to minimize grantee and participant burden, as well as decrease rates of attrition, data collected as part of this project will be collected immediately following each participant’s normally scheduled GPRA interview. SAMHSA requires all grantees to collect baseline GPRA data at intake, and follow-up data collection at six-months following the initiation of SAMHSA-funded services.

Participant responses will be recorded by project personnel on a hard copy of the Recovery Instrument provided in Attachment 2. If possible, SAMHSA hopes to incorporate the proposed Recovery instrument into its online data collection and reporting platform. If this effort is completed before data collection begins, grantee sites will be asked to input client responses into the online tool as part of their existing data reporting requirements. This web-based system will allow easy data entry, submission, and reporting to all those who have access to the system. Electronic submission of the data promotes enhanced data quality. With built-in data quality checks, easy access to data outputs and reports, users of the data can feel confident about the quality of the output. The electronic submission also promotes immediate access to the dataset. Once the data are put into the web-based system, it is available for access, review, and reporting by all those with access to the system from Center staff to the grantee staff.

If this task is not accomplished by the time that data collection begins, grantee sites will be asked to mail copies of the completed instruments to SAMHSA. Data will be entered into a secure database on CBHSQ’s internal servers and analyzed.

A4. Efforts to Identify Duplication and Use of Similar Information

The items collected are necessary in order to improve our assessment of clients’ level of recovery. SAMHSA is promoting the use of performance measures across all programs; this effort will result in less overlap and duplication, and substantially reduce the burden on grantees and clients that result from data demands associated with individual programs. The data collection proposed for these measures are not available elsewhere and is not duplicative. 

A5. Involvement of Small Entities

Individual grantees vary from small entities to large provider organizations. Every effort has been made to minimize the number of data items collected from programs to the least number required to accomplish the objectives of the effort and therefore, there is no significant impact involving small entities.

A6. Consequences of Collecting the Information Less Frequently

The proposed pilot project is a new data collection. The project will assess the usability and psychometric properties of the proposed tool among a voluntary group of SAMHSA grantees. Data will be collected at two time points – at client intake and at six-months post intake, two points in time during which SAMHSA grantees routinely collect data on the individuals participating in their programs to meet GPRA requirements for annual reporting.

Not collecting the data proposed as part of this project will prohibit SAMSA from being able to further refine its Recovery Measure. Consequently, the Agency will be unable to determine whether the Recovery Measure should be added to SAMHSA’s set of required performance measurement tools designed to aid in tracking recovery among clients receiving services from our funded programs. Failure to collect data will also prevent individual grantee sites from informing and enhancing client care.

A7. Consistency with the Guidelines in 5 CFR1320.5(d)(2)

This information collection fully complies with 5 CFR 1320.5(d) (2).

A8. Consultation Outside the Agency

The notice required by 5 CFR 1320.8(d) was published in the Federal Register on December 26, 2013 (78 FR 78373). No comments were received.

A9. Payment to Respondents

No monetary incentives are provided to grantees.

A10. Assurance of Confidentiality Respondents

SAMHSA’s grantees do not collect individually identifiable information for these programs. Only aggregated data will be reported by grantees, therefore, SAMHSA and its contractors will not receive identifiable client records.

A11. Questions of a Sensitive Nature

The proposed Recovery Measurement instrument includes questions on a number of potentially sensitive topics. Participants will be asked to report information on alcohol and drug use, as well as information on psychological symptoms. While the questions included in this instrument are no more sensitive than information SAMHSA is already collecting from participants as part of their GPRA interviews, some participants may experience feelings of minor discomfort or embarrassment when responding to these questions. Personnel at each grantee site are trained to deal with minor issues as they develop and provide additional services or referrals as needed.

A12. Estimates of Annualized Hour Burden

The total estimated respondent burden is 60 hours for the period from September 2014 through March 2015. Table 1 below summarizes the annualized respondent burden estimate.

The annualized cost to respondents is based on the latest publicly available data (May 2012) from the Occupational Employment Statistics Survey (OES), a mail survey that measures occupational employment for wage and salary workers in non-farm establishments in the US. The OES collects data from over 1.2 million business establishments through six semiannual panels over a three year period. It is sponsored by the Department of Labor, Bureau of Labor Statistics, and uses the OMB-required occupational classification system (the Standard Occupational System (SOC).

Per the OES, the mean hourly wage rate is $22.01/hour across all occupations. At an average wage rate of $22.01/hour and an average burden of 60 hours, the total average estimated 2014 and 2015cost is $1,320.60 or $660.30 per year. This estimated cost does not represent any out-of-pocket expense, but represents a monetary value attributed to the time spent completing the Recovery Measure instrument.

Table 1. Annualized Respondent Burden Hours, 2014 – 2015.

Estimated Annual Response Burden
Type of Grantees	Number of Respondents	Responses per Respondent	Average Hours per Response	Total Burden Hours	Hourly Wage1 ($)	Total Hours Cost ($)
Intake
Behavioral Health Treatment Court Collaborative (BHTCC)	100	1	0.10	10	22.01	220.10
Cooperative Agreements to Benefit Homeless Individuals (CABHI)	50	1	0.10	5	22.01	220.10
Primary and Behavioral Health Care Integration (PBHCI)	150	1	0.10	15	22.01	220.10
6-Month Follow-up
Behavioral Health Treatment Court Collaborative (BHTCC)	100	1	0.10	10	22.01	220.10
Cooperative Agreements to Benefit Homeless Individuals (CABHI)	50	1	0.10	5	22.01	220.10
Primary and Behavioral Health Care Integration (PBHCI)	150	1	0.10	15	22.01	220.10
Total	300			60		1320.60

¹ Department of Labor, Bureau of Labor Statistics, Occupational Employment Statistics Survey (OES), May 2012.

A13. Estimates of Cost Burden to Respondents

There are neither capital or startup costs nor are there any operation and maintenance costs.

A14. Estimates of Annualized Cost to the Federal Government

The estimated annualized cost to the Federal Government is $43,200. These costs are broken down in Table 2. Project management and data analysis will be performed by three CBHSQ employees. There will be no direct costs associated with data collection. Annual hours are based on a 40-hour work week for 48 weeks per year.

Table 2. Estimated Annualized Cost to the Federal Government.

Position	Estimated Annual Hours	Estimated Hourly Wage ($)	Total Annual Hours Cost ($)
Social Science Analyst, CBHSQ (Co-PI)	480	50	24,000
CBHSQ Contractor (Co-PI)	192	50	9,600
Social Science Analyst, CBHSQ	192	50	9,600
Total	864		43,200

A15. Changes in Burden

This is a new data collection.

A16. Time Schedule, Publication and Analysis Plans

It is anticipated that pilot study intake measures will be collected in September 2014. Six-month follow-up data will be collected in March 2015.

Information collected as part of this project will be used to directly impact SAMHSA’s performance measurement activities that are designed to support our ongoing Strategic Initiative focusing on recovery support.

Data collected as part of this project may also provide valuable information that may inform the recovery literature. SAMHSA plans to disseminate summary information collected from this pilot project through publications describing our Recovery Measurement project and Recovery Measurement instrument. These publications will not attempt to make any national estimates based on the information collected from this project. Instead, the publications will focus on the usability and the psychometric properties of the proposed measurement instrument.

A17. Display of Expiration Date

The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.

A18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

9