1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection of information.
The National Defense Authorization Act of Fiscal Year 2013 (NDAA) requires VA to “develop and implement a comprehensive set of measures to assess mental health care services furnished by the Department of Veterans Affairs” and to implement this initiative by December 31, 2013 (H.R. 4310-175/Sec. 726). The mandate further specifies that these measures will include “an accurate and comprehensive assessment of satisfaction of patients who receive mental health care services furnished by the Department.” As part of the response to this statutory mandate, Robert A. Petzel, MD, VA Undersecretary for Health, directed that the Performance Plans for all mental health senior leadership include the following objective: “Develop mental health quality measures that are population based and outcome oriented.”
Immediately following passage of the NDAA, efforts began to identify existing measures that provide population-based measures of outcomes, as well as additional measures that would be needed to fulfill the NDAA mandate. Using the framework developed by the National Quality Forum , a workgroup identified measures that are currently available in VA or could be readily derived from existing data sources. The workgroup concluded that there are presently no sources of systematic data regarding patient satisfaction specific to their mental health treatment or reflecting changes in patients’ symptoms and functioning following initiation of treatment. For example, the Survey of Health Experience of Patients (SHEP) has been developed to measure patient satisfaction in the Veterans Health Administration and has been in use since 2008, but this survey is not specific to mental health treatment and does not include questions related to key mental health outcomes. Locally administered satisfaction surveys do not provide comparable data across programs nationwide, tend to under represent individuals who drop out of treatment, and do not measure mental health outcomes.
The workgroup therefore developed a plan for a brief survey of a representative sample of new mental health treatment patients. The survey is designed to fulfill the NDAA mandate and to provide data on the impact of mental health treatment programs on patients’ mental health symptoms, functioning, and satisfaction. The Veterans’ Outcome Assessment (VOA) comprises 42 items with a burden of 30 minutes. These items and their sources are as follows:
General Health and Functioning
CDC HRQOL-14 Healthy Days Measure/Healthy Days Core Module (Items #1-4)
WHO Disability Assessment Schedule 2.0 (WHODAS 2.0) (Items 5-8)
Symptoms, Behaviors, and Well-Being
Kessler 6 (Items #9-14)
Brief Addiction Monitor (Items #15-16)
Schwartz Outcomes Scale-10 (Items #17-26)
Veteran Recovery Assessment (Items 27-31)
Illness Management and Recovery Scales (Items #32-33)
Global Quality of Life Scale (Item #34)
Patient Global Impression of Improvement Scale (Item #35)
Experience with VA Health Care
Experience of Care and Health Outcomes Survey (Items #36-39)
TCU Treatment Engagement/Treatment Satisfaction Scale (Item #40)
Assessing for Suicidality
Patient Health Questionnaire (PHQ-9) (Item #42)
The VOA will be administered by a call center to a sample of 400 Veterans seeking mental health treatment each quarter. The sample will be randomly selected from patients with a new episode of mental health treatment, defined as those patients who have had no mental health treatment in the past 9 months. The sample will be stratified in order to ensure adequate representation for each Veterans Integrated Service Network (VISN). Patients will be called as soon as possible (generally within a week) after initiation of treatment and 3 months later and will include patients who remain in treatment and those who drop out.
The sample size was determined to allow for reliable statistical results at the national level on a quarterly basis and at the network level on a yearly basis. Survey responses will be analyzed and posted on the VHA Support Service Center website for field use or posted as special reports. Key summary findings will be included in the required NDAA reporting.
2. Indicate how, by whom, and for what purposes the information is to be used; indicate actual use the agency has made of the information received from current collection.
The mental health outcomes information obtained through this new collection will be used by VA leadership, including those in the Offices of Mental Health Operations and Mental Health Services , Network offices, and VA Medical Centers. Such information on Veteran mental health outcomes is crucial to guide resource allocation and programmatic decisions for mental health programs and to intervene effectively to prevent individual adverse outcomes such as suicide, overdose deaths, and morbidities associated with mental illness and to support recovery-oriented treatment designed to improve functioning and reduce symptoms. The data will allow VA policy makers to reliably track national performance on a quarterly basis and to track VISN performance on a yearly basis. These data will reveal trends in outcomes over time and will help in pinpointing programs that are doing well in terms of patient outcomes, so that other programs can emulate their practices, as well as identifying those programs that are performing poorly so that steps can be taken to improve them. Results of the survey will be reported to Congress and will influence decisions on funding. The VOA will thus provide Veterans who are experiencing mental health problems with a direct voice in program evaluation and improvement. Summary data on performance also will be available on a public website, as mandated by the NDAA, to provide Veterans and their families with additional information for purposes of managing their mental health treatment and U.S. citizens with information regarding VA’s mental health programs and Veterans satisfaction with their care.
3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.
The planned project will use a telephone call center and computer-assisted telephone interviews to reach a national sample of Veterans at initiation of mental health treatment and 3 months later. Consideration was given to use of other technological collection techniques, but none would allow for contacting patients who drop out of treatment or for efficiently following up with patients after 3 months. Telephone administration of the survey will allow Veterans to be contacted in a timely manner after their initial mental health appointment. A computer-based survey tool will allow the responses Veterans make over the phone to be entered immediately and electronically and will reduce the Veteran respondents’ time and effort and ensure data security and confidentiality. The interview will inform respondents of the requirements of the Privacy Act and the Paperwork Reduction Act, as noted in the script constituting the information collection instrument.
4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.
Immediately following passage of the NDAA, a workgroup was convened to identify existing measures that provide population-based information on outcomes and satisfaction in mental health treatment. The workgroup examined a number of potential sources of duplication and concluded that none of the already available data sources and information could serve the purposes mandated by the NDAA.
The closest data sources and rationale for exclusion include the following:
SHEP – This survey includes all outpatients and is a broad survey asking generic questions about outpatient visits. It does not focus on mental health patients and is not specific to mental health outcomes, mental health treatment, or satisfaction with mental health care.
Mental Health Assistant – These data are collected by providers through clinical reminders during appointments. The data are problematic for assessing Veteran outcomes because they come only from patients who attend appointments (dropouts are not assessed) and from patients whose providers choose to complete them to help guide individual care decisions. Furthermore, satisfaction data, a key mandate of the NDAA, are not obtained.
Data from research studies conducted in VHA do not permit systematic, nationwide, long-term tracking of program outcomes.
The workgroup consulted with the RAND Corporation, Department of Defense, and Group Health Cooperative regarding potential outcome measures and concluded that in order to assess the range of outcomes (functioning, symptoms, satisfaction) without undue burden on participants, it would be necessary to draw items from a number of existing measures. A recent publication reporting an external evaluation of VA mental health programs similarly identified no adequate source of mental health outcome data for patients in these programs (Watkins, Pincus, Paddock, Smith et al., 2011).
Watkins, K. E., Pincus, H. A., Paddock, S., Smith, B., Woodroffe, A., Farmer, C., & Call, C. (2011). Care for veterans with mental and substance use disorders: good performance, but room to improve on many measures. Health Affairs, 30(11), 2194-2203.
5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.
No small businesses or other small entities are impacted by this information collection.
6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.
If the collection is not conducted, VA will not be compliant with the congressional mandate of the NDAA of Fiscal Year 2013 and will not be able to fulfill its general mandate to improve health care for Veterans through systematic program evaluation. If collection is conducted less frequently than the proposed 3-month follow up, it will be more difficult to distinguish the impact of the mental health treatment program from other influences in the Veteran’s life. The length of the survey and proposed sample size have been reduced as much as possible while still maintaining the required scope of the survey, adequate measure reliability, and reliable data for comparisons over time and between organizational units. Without the proposed collection, Congress, VHA leadership, and local leadership will be unable to evaluate outcomes of mental health services/care for Veterans having an episode of mental health care. These parties will have little empirical evidence on which to determine whether recent mental health initiatives (increased mental health staffing, peer support, community-collaboration, site visits, etc.) result in outcome improvement at the regional and national levels.
7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.
There are no such special circumstances. None of the special circumstances described above are applicable to this project.
8. a. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.
The notice of Proposed Information Collection Activity was published in the Federal Register on March 24, 2014 (Volume 79, Page 16102). We received no comments in response to this notice.
b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and recordkeeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances which preclude consultation every three years with representatives of those from whom information is to be obtained.
A number of people from within and outside the VA were consulted during the development of the project. As noted above, the workgroup consulted with RAND, Department of Defense, and Group Health Cooperative regarding potential outcome measures. In developing the survey itself, input was sought on the frequency of collection, clarity of instructions, and necessary data elements. The experts consulted included:
Paul Crits-Christoph, PhD; Director of the Center for Psychotherapy Research, University of Pennsylvania
Helena Kraemer, PhD; Professor Emerita of Biostatistics in Psychiatry, Department of Psychiatry and Behavioral Sciences, Stanford University; Professor of Psychiatry, Department of Psychiatry, University of Pittsburgh School of Medicine
Heather Law; Research Programme Co-ordinator, Greater Manchester West Mental Health NHS Foundation Trust, Manchester, United Kingdom; School of Psychological Sciences, University of Manchester
Priscilla Ridgway, PhD; Community and Organizational Researcher, Center for Community Support and Research at Wichita State University.
Diana Rofail, PhD; Global Head of Patient Reported Outcomes (CNS & Metabolism), Roche.
9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.
No payment or gift is provided to respondents.
10. Describe any assurance of privacy to the extent permitted by law provided to respondents and the basis for the assurance in statute, regulation, or agency policy.
Survey instructions specify that responding to the survey is completely voluntary, confidential to the full extent of the law, and will have no effect on entitlement to or eligibility for VHA healthcare benefits. The VHA follows the requirements of the Privacy Act and adheres to U.S.C. 38, Section 5705, Confidentiality of Medical Quality-Assurance Records. The survey will include a Privacy Act statement to each respondent that the information is being solicited pursuant to the Congressional mandate in the NDAA, that participation is voluntary, that failure to participate will not compromise an individual’s receipt of VA benefits, and that no individually identifiable information collected will be reported.
Data will be kept in electronic form. Electronic data will reside on secure VA password-protected servers behind the VA firewall. All data transfers will occur within the VHA intranet. Access to these electronic folders will be authenticated through VA login procedures. Data management and security will follow VA data security policies and other applicable Federal/State laws and regulations. Access to data will be granted only to VA employees explicitly authorized to work on the project. These employees have been appropriately credentialed and have completed the information privacy training requirements. Employees will have access to only the data as they need to perform their duties.
11. Provide additional justification for any questions of a sensitive nature (Information that, with a reasonable degree of medical certainty, is likely to have a serious adverse effect on an individual's mental or physical health if revealed to him or her), such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
By virtue of its purpose to assess of mental health treatment outcomes, the survey includes questions regarding symptoms and functioning that are of a potentially sensitive nature. These questions are essential to accomplishing the mandated assessment of VA mental health treatment programs. The script containing the actual survey questions is included as a part of this collection request. For example, participants are asked about their use of any illegal drugs in the past 30 days in order to determine whether VA mental health treatment programs are effectively addressing Veterans’ current substance use problems. Similarly, they are asked about suicidal thoughts in order to determine whether VA mental health treatment programs are effective in reducing suicide risk.
In order to encourage Veterans to provide answers to these potentially sensitive questions, the survey introduction underlines the ways in which this information can benefit Veterans using mental health services and help VA to improve these services. Potential participants are told, “You do not have to answer any question you do not want to answer and your decision to participate and the answers you give will have no effect on your entitlement to or eligibility for VHA healthcare benefits, and will not affect the VA services you receive.” Participants also are assured that no individual responses will be reported but that all such reports will summarize the results nationwide or for groups of patients in particular VA medical centers and networks. Beyond those conducting the survey and carrying out analyses, the individual identities of participants will not be known or disclosed.
12. Estimate of the hour burden of the collection of information:
Burden was estimated by timing the completion of interviews done in previous administrations of a similar survey and in pretests (using 9 or fewer persons). Our goal is to collect baseline and follow-up surveys from 400 patients per quarter. Our sampling estimates are based on previous experience with the Veterans Health Outcomes Improvement Project , a similar call-center based outcomes collection pilot project, which found about 40% of all calls resulted in contact with a Veteran, 50% of all contacts resulted in a Veteran starting a baseline survey, and about 60% of baseline participants provided follow-up surveys 3-months later. Based on this experience, we estimate that in order to collect baseline and follow-up surveys from 400 patients each quarter, we will need to make about 3300 calls, make contact with 1200 Veterans, and administer about 670 baseline surveys each quarter, with successful follow-up completion with 400 respondents. On a yearly basis, time estimates are as follows:
The number of respondents, frequency of responses, annual hour burden, and explanation for each form is reported as follows:
VA Form 10-TBD |
Number of respondents/ year |
x Hours/ respondent |
Equals Total Number of Hours |
Veterans Outcome Assessment - Baseline |
2,680 |
.5 |
1,340 |
Veterans Outcome Assessment – Follow Up |
1,600 |
.5 |
800 |
Total |
4,280 |
|
2,140 |
b. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB 83-I.
This request covers only one form.
c. Provide estimates of annual cost to respondents for the hour burdens for collections of information. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14.
VA does not require any additional record keeping. The cost to respondents for completing these forms is $51,360 ($24 per hour x 2,140 burden hours).
13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).
a. There is no capital, start-up, operation or maintenance costs.
b. Cost estimates are not expected to vary widely. The only cost is that for the time of the respondent.
c. There is no anticipated recordkeeping burden.
14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.
The annual cost to the Government is estimated as $668,800. This cost includes survey management, questionnaire design, creating a computerized battery, baseline and 3-month follow-up data collection, data cleaning, data analysis, and dissemination of results.
15. Explain the reason for any burden hour changes or adjustments reported in items 13 or 14 of the OMB form 83-1.
This is a new collection and all burden hours are considered a program increase.
16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.
VA does not intend to publish these data in peer reviewed journals or reports. Key summary findings will be included in public-facing and Congressional reports as required by NDAA. Results of the survey will be made readily available to VACO, VISN, VHA field staff and stakeholders via the VA Intranet and in executive summary reports. Upon request, information will be made available to concerned program officials, OMB, Congress, Veterans' Service Organizations (VSO), the news media, and interested citizens through the Freedom of Information Officer.
17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.
The survey will be administered by telephone and computer data entry. The expiration date will appear on the survey instrument.
18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.
We do not have any exceptions in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB form 83-I.
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Bold black = OMB questions |
| Author | vhacobickoa |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |