Statement A 07_17_13

Supporting Statement For

Veterans Benefits Administration (VBA)

Voice of the Veteran (VOV) Surveys

(2900-0782)

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

In 2008, VBA recognized a need to develop and design an integrated, comprehensive Voice of the Veteran (VOV) measurement program for its lines of business: Compensation Service (CS), Pension Service (P&F), Education (EDU) Service, Loan Guaranty (LGY) Service and Vocational Rehabilitation and Employment (VR&E) Service. This continuous measurement program will help VBA understand what is important to Veterans relative to benefits and services provided by VBA and will provide VA/VBA leadership with actionable and timely Veteran feedback on how VBA is performing against those metrics. Insights will help identify opportunities for improvement and measure the impact of improvement initiatives.  

VBA conducted a benchmark study in Fiscal Year 2013 (October 2012 through January 2013) in order to validate the survey instruments, identify Key Performance Indicators, and establish performance benchmarks. Findings and recommendations are pending review and approval.

Based on interviews conducted, J.D. Power and Associates (JDPA) has separated the Veterans experience with the VBA into two categories – Enrollment in a Benefit and Servicing of a Benefit. Stakeholders expressed the need to understand the different elements that make up the overall experience with VBA for the Veteran and wanted to ask questions relevant to each specific population. The following outlines how that will be approached with each of the lines of business.

J. D. Power will be fielding three survey instruments for CS & P&F lines of business. Based on the numerous interviews conducted, there will be one survey instrument for the Enrollment category that will be used for both compensation and pension claimants; while compensation beneficiaries and pension beneficiaries will receive separate Servicing instruments. The survey pool for the VOV Tracking Study CS & P&F Enrollment questionnaire will include individuals who have received a decision on a compensation or pension benefit claim within 30 days. Specifically, the sample will be stratified as follows: compensation enrollment will include individuals who have received a decision in the past 30 days (includes those who were found eligible on a new or subsequent claim and those who have been denied and are not appealing the decision) and pension enrollment will include individuals who have received a decision in the past 30 days (includes those who were found eligible on a new or subsequent claim and those who have been denied and are not appealing the decision).

The survey pool for the tracking study for the Compensation Servicing questionnaire will include individuals who received a decision or are receiving benefit payments. The survey pool for the tracking study for the Pension Servicing questionnaire will include individuals who have currently been receiving benefits for at least 6 months.

J.D. Power will be fielding two survey instruments for Education Service. The survey pool for the tracking study for the Education Enrollment questionnaire will include individuals who have received a decision on their education benefit application within 90 days (i.e., the original end-product has been cleared within the past 90 days) prior to the fielding period. The sample will be stratified as follows: (1) accepted and enrolled, (2) accepted and not enrolled. The survey pool for the tracking study Education Servicing questionnaire will include beneficiaries who have been enrolled and receiving benefit payments for at least 2 consecutive school terms.

J.D. Power will be fielding two survey instruments for Loan Guaranty Service. The survey pool for the tracking study for the LGY Enrollment questionnaire will include individuals who closed on a VA home loan in the 90 days. The sample will be stratified as follows: (1) those who closed on purchase loans, (2) those who received loans for interest rate reductions, and (3) those who obtained cash out or other refinancing. The survey pool for the tracking study for the SAH Servicing questionnaire will include individuals who are eligible for a specially adapted housing grant and in the past 12 months have: (1) received an approval on their grant and are currently somewhere in post-approval, (2) have had all their funds dispersed and final accounting is not yet complete, and (3) have had all of their funds dispersed and final accounting is complete.

J.D. Power will be fielding three survey instruments for Vocational Rehabilitation & Employment Service. The survey pool for the tracking study for the VR&E Enrollment questionnaire will include individuals who had an initial meeting with their VR&E counselor and were granted a decision regarding their entitlement in the past 30 days prior to the fielding period. The sample will be stratified as follows by those who applied/applied and show up for initial appointment and: (1) were entitled to program and pursued, (2) were entitled to program but do not pursue, (3) were not entitled to program. The survey pool for the tracking study for the VR&E Servicing questionnaire will include individuals who have entered and been enrolled in one of the five tracks for at least 60 days prior to the fielding period. The sample will be stratified as follows: (1) Veterans who have been rehabilitated and (2) Veterans who have reached maximum rehabilitation gain and could not proceed in program. The VR&E Non-Participant survey will explore why eligible individuals did not take advantage of their benefit entitlement. The survey pool for the tracking study for the VR&E Non-Participant questionnaire will include individuals who dropped out of the program prior to completing a rehabilitation plan. The sample will be stratified as follows: (1) applicants who never attended the initial meeting with a counselor and (2) applicants who started, but did not complete a rehabilitation plan (i.e., negative closures).

This information collection request is for a tracking study for the VOV program. The results of the tracking study will allow VBA to measure the effectiveness of new initiatives and changes in processes identified within the VOV Tracking Study. Further, the volumes for each survey-type have been increased in order to provide regional offices with a statistically valid sample size, as approved and reviewed by VBA Leadership and statistician.

These surveys will be conducted per the legislative requirement set forth in the Executive Order 12862 and section 5701 of Title 38 of the United States Code. The Executive Order mandates that government agencies set standards and seek to measure agency performance against such standards. These surveys are also being conducted in concurrence with the Government Performance Results Act (GPRA) and its requirements for improving customer service.

Specifically, VBA requires customer satisfaction information from Veterans in order to track the effectiveness and usability of the benefits offered, as well as to assess the level of satisfaction with the level of service provided by VBA. The data collected will be used by VBA to make improvements to the benefit application and enrollment process, which in turn, will enable VBA to maximize the efficiency and quality of service provided to its nation’s Veterans and their dependents.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from current collection.

The information to be collected with the planned surveys will be used by VBA leadership and the lines of business: Compensation Service (CS), Pension Service (P&F), Education (EDU) Service, Loan Guaranty (LGY) Service, and Vocational Rehabilitation and Employment (VR&E) Service to determine Veteran satisfaction with benefits and services. The major use of this data will be three-fold: 1) to provide Veterans an opportunity to comment on their experience with benefits and services, 2) to formulate program and policy changes in the enrollment and servicing process to ensure the most effective delivery of benefits and services, and 3) to identify the most influential elements of Veterans’ experience on their satisfaction and establish performance benchmarks to gauge the effectiveness of program and policy changes in improving Veterans’ experience with benefits and services. Currently, there are no other means of continuously evaluating Veteran satisfaction with VBA’s CS, P&F, EDU, LGY, and VR&E benefits and services.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

In conjunction with a contractor, VBA staff will develop, administer, and analyze the Compensation and Pension Enrollment surveys, Compensation Servicing survey, Pension Servicing survey, Education Enrollment and Servicing surveys, the LGY Enrollment and SAH Servicing surveys, and the VR&E Enrollment, Servicing, and Non-Participant surveys.

To provide flexibility to the Veteran, the surveys will be self-administered in both paper form and online using web-based data collection. This will maximize the timeliness, efficiency, and response rate of data collection from Veterans. Based on previous experience, it is expected that the online web-based data collection will yield a 10% completion rate, with 20-25% responding via the mail option – a target of 30-35% response rate.

VBA has selected a contractor that has strong capabilities in programming and hosting surveys on the worldwide web, and maintains effective security and privacy procedures (e.g., unique passwords for respondents, data encryption) when designing and programming web surveys. The web address (URL) on which the surveys will be hosted and accessed by respondents will be included in the materials sent to Veterans advertising the surveys. Veterans can either connect to the appropriate web page and complete the survey or fill out the survey included in the materials packet.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

As noted, VBA has not recently conducted customer satisfaction surveys specifically geared toward Veteran satisfaction with CS, P&F, EDU, LGY or VR&E benefits and services on a continuous basis since 2005.

Our review of available data suggests that no sources outside of VBA’s VOV survey program are available that provide a reliable, representative, and statistically valid sample of the targeted audience who have used CS, P&F, EDU, LGY or VR&E services and from which detailed customer satisfaction indicators can be derived.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If the Voice of the Veteran tracking surveys are not conducted, VBA will not be able to fulfill its performance measurement requirements as articulated by the requirements of Public Law 103-62, the Government Performance and Results Act of 1993 and Title 38 USC, §527, Evaluation and Data Collection. Furthermore, VBA would miss an opportunity to learn from America’s Veterans about how to improve the CS, P&F, EDU, LGY or VR&E benefit enrollment and servicing processes to best serve their needs. VBA would also lose an opportunity to track and document improvements or declines in Veteran satisfaction and service delivery over time.

The design and administration of the suite of VOV surveys incorporates significant measures to minimize burden on respondents. (These specific measures are discussed in more detail in section 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and are intended to produce valid and reliable results that can be generalized to the universe of study.

8. Part A: If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

A 60-day Federal Register (FR) notice was published on June 20, 2013, for the Voice of the Veteran Benchmark Study, pages 37278-37288. There were no comments received in response to this notice.

8. Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances which preclude consultation every three years with representatives of those from whom information is to be obtained.

VBA is not aware of any tracking surveys in private or government sectors.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift shall be provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

An assurance that the information will be kept private to the extent of the law is made in the introductory letter respondents receive with the survey as well as written prologue appearing at the beginning of each survey, regardless of the mode of administration. Respondents are assured that answers given will be kept private to the extent of the law and will not affect their current or future entitlement to benefits and services. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code).

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

None of the survey instruments contain any questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information.

Title	Number of Respondents	Total No. of Responses	Completion Time
Compensation Enrollment % Collected Electronically	36,000 12,000	1 1	0.25 0.25
Compensation Servicing % Collected Electronically	13, 500 4,500	1 1	0.25 0.25
Pension Enrollment % Collected Electronically	3,000 0	1	0.25
Pension Servicing % Collected Electronically	3,000 0	1	0.25
Education Enrollment % Collected Electronically	3,600 1,200	1 1	0.25 0.25
Education Servicing % Collected Electronically	2,250 750	1 1	0.25 0.25
VR&E Enrollment VR&E % Collected Electronically	13, 500 4,500	1 1	0.25 0.25
VR&E Servicing % Collected Electronically	18, 000 0	1	0.25
VR&E Non-Participant Ben. % Collected Electronically	1, 500 0	1	0.25
Loan Guaranty % Collected Electronically	9,000 3,000	1 1	0.25 0.25
Specialty Adapted Housing Grant % Collected Electronically	1,500 0	1	0.25
Total:	130,800

There are 130, 800 responses, it takes 15 minutes to complete the form, which calculates to 32,700 hours of Respondent Burden.

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

a. There is no capital, start-up, operation, or maintenance costs.

b. Cost estimates are not expected to vary widely. The only true cost is that for the time of the respondent (average of 15 minutes per respondent). Using the Federal minimum wage of $7.25 an hour and the estimated 15 minute per respondent time burden, the total monetized estimate of time is $237,082.25 annually. Based on the anticipated duration of the program for the next three years and the current Federal minimum wage of $7.25, the total sum is $711,246.75.

c. There are no anticipated capital start-up cost components or requests to provide information.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government for the tracking study program is estimated at $1,525,847.60 per year, or $4,577,543 for the next three years. Table 2 below shows the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to a contractor and will be included in the contractor’s total cost. This cost is determined based on the cost per hour per number of respondent’s per minutes - divided by sixty.

Cost Item	Hours	Cost
VA-Labor	4,500	$209,765
Contractor Costs		$4,367,778
Total	4,500	$4,577,543

The VA Labor cost was estimated using a composite average salary and benefits figure of $45.51 per hour for 4000 hours at a GS-13, step 3 and $55.45 per hour for hours at a GS14, step 4.¹ The amount paid to the contractor for the survey effort includes as its major components the survey of recent claimants or existing beneficiaries for a projected total cost of $1,525,847.60 on an annual basis for the next three years. For the tracking program, these costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for Web administration, questionnaire pretest, validation, data processing, providing a clean data file, project management and analysis, and reporting.

15. Explain the reason for any changes reported in Items 13 or 14 above.

Survey volumes and expected returns have increased nearly 300% in order to support VBA’s VOV Continuous Tracking Study. The volumes are being increased in order to provide regional offices with a statistically valid sample size, as approved and reviewed by VBA Leadership and statistician.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Based on previous survey experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.

16A: TABULATION AND PUBLICATION

• Scan Surveys and Monitor Response Rates

During the survey field-period, the surveys will be scanned as they are received to ensure the currency of the name and address data file. During the scanning process, VA’s contractor will inspect and remove duplicate surveys in case any individual inadvertently received a second survey after completing one from the first mailing. Data from damaged surveys could potentially be lost unless appropriate actions are rendered to make the data usable. Staples, paper clips, and notes will be removed and stray pencil marks that would interfere with the scanning process will be erased.

The contractor will host a secure password-protected file transfer protocol (FTP) site that will provide response rates from the survey. Throughout the period that the surveys are being fielded, response rate data will be updated at a minimum on a weekly basis for the survey. CS, P&F, EDU, LGY and VR&E staff will be provided with passwords to access the site at any time and receive formal training on the use of the site.

• Quarterly Program Line Briefings

Program lines will receive an overview of findings that outline the following:

• Summary of key score matrices

• Key attributes

• Key diagnostics and demographics

• Verbatim (open-ended) response report

The national level report for each business line will contain a summary of business line specific results, as well as a summary comparison of all business lines for measures that are common across business lines.

VBA’s contractor will pre-test automation routines for new designs in the analysis to ensure that the report exhibits graphics and data tables that can be reliably and accurately programmed for each report type. Development of the automated report system will consist of several concurrent tasks, including a data crosswalk of the survey iterations to map identical variables and response sets, Statistical package for the Social Sciences (SPSS) syntax routines and development of prototype report templates using Microsoft Excel with Visual Basic for Applications.

• Conduct Final Briefing

The contractor will develop data tabulations on survey highlights from the survey and present the results to CS, P&F, EDU, LGY, VR&E and VBA staff. The contractor will work closely with VBA to determine the most salient findings to present in the briefing. The contractor will deliver bound color copies of the briefing presentation to CS, P&F, EDU, LGY and VR&E.

• Load Data to Reporting Site

Data collected will be loaded to the VOV Reporting Site for all program line and regional offices. Reporting Site was developed under a different order.

• Deliver CD of Final Products

The contractor will submit the final deliverable of CDs containing electronic files as specified in the SOW. Documents and materials will include:

The contractor shall provide the following files (in MS Excel format) on CD or secure FTP site: a) the names and addresses of beneficiaries who were selected for the survey, but who were determined through the course of the survey to have moved, or have undeliverable addresses; (b) the names, and addresses of any veterans who were reported as deceased, or who were reported to have ‘undeliverable’ addresses, delineated by type of survey (c) the names and addresses of all veterans who were sent a questionnaire during this survey iteration; (d) the names and addresses of all non-respondents, broken out by type of survey they were selected for.

• Raw survey data on CD in SP&FS format

• Copies of the reports in electronic format on CD

• Copies of the questionnaire and survey mailing materials

• Survey respondent verbatim, grouped by Regional Office/Line of Business

• Raw respondent data for undeliverable addresses, deceased respondents, non-responders, and a list of all respondents who were selected for mailing

These CDs of electronic files will provide CS, P&F, EDU, LGY and VR&E with all materials used during the project, along with the final reports.

16B: PROJECT TIMELINE

The major activities for the VBA VOV survey project are structured by task, and are outlined below.

• Task 1: Conduct kickoff meeting and develop the project management plan (PMP)

• Task 2: Monthly Status Meetings and written summaries

• Task 3: Methodology and Sampling Plans, and Production Schedule

• Task 4: Sample cleaning, Imaging and Data Collection, and Disposition Reports

• Task 5: Host live survey websites and Disposition Reports

• Task 6: Host toll-free telephone and web helpdesk assistance and corresponding call/inquiry logs

• Task 7: Data Matrices, Verbatim Reports, Data Analysis

• Task 8: Data Load to VOV Reporting Site and Training Sessions

• Task 9: Business line analysis and presentations

• Task 10: Semi-Annual Leadership Briefings

• Task 11: Action Plan development and sessions

TIMELINe – VOICE OF THE VETERAN SURVEYS

Conduct kickoff meeting and develop the project management plan (PMP)	May 2013
Monthly Status Meetings and written summaries	Continuous
Methodology and Sampling Plans, and Production Schedule	May 2013
Sample cleaning, Imaging and Data Collection, and Disposition Reports	Continuous
Host live survey websites and Disposition Reports	Continuous
Host toll-free telephone and web helpdesk assistance and corresponding call/inquiry logs	Continuous
Data Matrices, Verbatim Reports, Data Analysis	Continuous
Data Load to VOV Reporting Site and Training Sessions	Continuous
Business line analysis and presentations	Continuous
Semi-Annual Leadership Briefings	November 2013 and May 2014
Action Plan development and sessions	August 2013 and March 2014

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The collection survey instrument may be reproduced and/or stocked by respondents and Veterans organizations. Requiring VA to display an expiration date on the form would result in unnecessary waste of existing stocks. Inclusion of the expiration date could also result in a delay of the Department's action on the benefit being sought (respondent trying to obtain a newer version, while VA would have accepted the older version). For the reasons stated, VA continues to seek an exemption that waives the displaying of the expiration date.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

1 FY 2013 Budget Estimates

11