4 BVA_Appellant_Justification A

Supporting Statement A

For

Board of Veterans’ Appeals

Voice of the Veteran Appellant Surveys

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The mission of the Board of Veterans’ Appeals (Board) is to hold hearings and issue timely quality decisions on appellate matters.

Currently, the Board collects customer satisfaction on a very limited basis. Surveys are distributed after the hearing is conducted relying on respondents to mail in the postcard. The survey card only measures the appellant’s satisfaction with the hearing process and response rates are low. The Board will benefit from obtaining direct feedback from Veterans regarding their experience with the Board with either the hearing or non-hearing experience. Specifically, the Veterans’ feedback will provide the Board three key benefits: 1) identify what is most important to Veterans in determining their satisfaction with both the hearing and non-hearing process; 2) determine what to do to improve experience; and 3) serve to guide training and/or operational activities aimed at enhancing the quality of service provided to Veterans.

The Board and the Contractor will survey Veterans who have had their appeal decided either through a hearing or non-hearing experience. This will enable the Board to gauge the effectiveness of the Board’s process delivering information and assistance to Veterans, as well as assess Veterans’ overall level of satisfaction with the Board experience. In addition, the data will be used by the Board to make potential improvements to the Board’s operational processes and service delivery, which in turn, will enable the Board to serve Veterans in the most efficient and effective way possible.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from current collection.

The information collected during the planned FY13 surveys will enable the Board to understand and quantify satisfaction levels of Veterans who have gone through the benefit appeal. Since currently satisfaction is only measured via a survey card that is completed after the hearing process, these results will offer the Board critical inputs that can be used to assist decision makers to formulate operational changes in the Board environment to ensure that Veterans are effectively served.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The appellant satisfaction survey is composed of two separate surveys. The phone portion will be a brief assessment of appellant satisfaction with the hearing process or non-hearing process and tailored to collect email addresses for the more-in-depth online survey. Email invitations will be sent weekly to those appellants who provide an email address during the phone interviews. The e-Survey instrument will measure various processes an appellant undertakes with the Board when appealing a decision. Data collection will occur during quarterly time frames.

J.D. Power and Associates will conduct the initial phone interview among Veterans who have completed the appellant process. Using sample files provided by the Board, the Contractor shall randomly select an adequate sample to complete 14,727 interviews (combination of eSurvey and telephone interviews).

Telephone calls will be made between 11 a.m. and 4 p.m. on Saturdays (no telephone calls on Sunday) and between 2 p.m. and 7 p.m. on weekdays in the potential respondent’s local time zone. Telephone calls will not be made on designated holidays. A maximum of seven call attempts will be made to each potential respondent. Interviews will be conducted by live interviewers who will inform potential respondents who they are and why they are calling at the beginning of each call.

In conjunction with the Contractor, the Board staff will develop, administer, and analyze the survey data. Also, the Contractor in collaboration with the Board will select the telephone survey provider(s) that will maintain effective security and privacy procedures (e.g., data encryption).

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

Currently the Board collects Veterans’ satisfaction on a limited basis. Specifically, The Board has created VA Form 0745, Board of Veterans’ Appeals Hearing Survey Card, which is a customer comment card provided to the appellants upon completion of their hearing. The card is postage paid and the appellant is instructed to fill it out and mail it in. This Survey Card only measures the appellant’s satisfaction with the hearing process only and relies on the appellant to mail the card back in. The collection process is limited and the response rate is very low. The Contractor is designing a program that will be more inclusive, including non-hearing appellants and will reach out to all appellants. The survey instrument is designed as to not duplicate questions that are being addressed in the Survey Card. Additionally, review of available data suggests that there are no outside sources of data which the Board could use to obtain a representative sample of the Veterans feedback on appellant satisfaction.

To prevent duplication within the survey (i.e. calling the same appellant more than once) the Board and the contractor will de-dupe the call list so that an appellant is only contacted after their most recent appeal has been adjudicated. Appellants with the same telephone number will not be contacted more than one time in a six-month time period.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If the Board is unable to proceed with the collection of the data, it will not have the benefit of receiving appellant feedback on what is important to them or how best to improve their service and to best serve the needs of our Veterans. The collection of the data will enable the Board to track and document improvements or declines in the appellant process over time.

The design and administration of the appellant satisfaction survey incorporates significant measures to minimize burden on respondents (see Item 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and will produce valid and reliable results that can be generalized to the universe of study.

8. Part A: If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

In compliance with 5 C.F.R. 1320.8(d), VA solicited comments from members of the public and affected agencies concerning the proposed new information collection “Board of Veterans’ Appeals, Voice of the Veteran Appellant Surveys.” See 78 Fed. Reg. 54956-57 (September 6, 2013).

Several errors appeared in the September 6, 2013, Federal Register Notice in paragraphs pertaining to “burden hours”. The errors occurred due to non-use of converting hours into minutes. The “Frequency of Response” was changed from “Quarterly” to “Annually”. However, these errors only appeared in the Federal Register Notice and were inconsistent with that that was reported in the Supporting Statement. A corrected Federal Register Notice was published to reflect these corrections. See 78 Fed. Reg. 60379 (October 1, 2013)

VA received one comment from the public. The comment did not specifically address whether the proposed collection of information is necessary for the performance of VA’s functions; the accuracy of VA’s estimates of the burden of the proposed collection of information; the quality, utility, and/or clarity of the information to be collected; or ways to minimize the burden of the information collection on those who are to respond. Therefore, no further action will be taken with regard to the proposed information collection request for the “Board of Veterans’ Appeals, Veterans, Voice of the Veteran Appellant Surveys in response to the comment received.

8. Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances, which preclude consultation every three years with representatives of those from whom information is to be obtained.

The Board’s Contractor for this survey is J.D. Power and Associates. The Board is consulting with J.D. Power and Associates regarding their Voice of the Consumer (VOC) satisfaction expertise. Their ability to provide benchmarked data offers the Board a set of well-established and accepted industry performance norms across numerous industries from which to compare and contrast various reporting metrics.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift shall be provided to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

An assurance of strict confidentiality is made in the introduction respondents receive with the live telephone interviewer. Respondents are assured that answers given will be kept private to the intent of the law and will be used for research purposes only. Respondents will also be given the opportunity to opt-out of completing the survey during the introduction. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code). The Privacy notification is included in a list of frequently asked questions that respondents have. During the interview, J.D. Power responds as follows:

Question: Will my responses during this interview remain confidential?

Answer: Yes, your responses during this interview will remain completely confidential and will not affect your eligibility for current or future benefits. The answers you provide will not be linked to your name or contact information.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The survey instrument does not contain any questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information.

TABLE 1: ESTIMATED ANNUALIZED TIME BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Number of responses per respondent	Average burden per response (in hours)	Total burden hours
Veterans who have received a decision from the Board (phone survey -80%)	11,782	1	.08	982
Veterans who have received a decision from the Board (E-survey -20%)	2945	1	.20	589
Total Hours				1571

TABLE 2: ESTIMATED MONETARY BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Estimate hourly wage	Average burden per response (hr.)	Total cost to all respondents
Veterans who have contacted the Board Call Center	14,727	24	.14	49,482
Total				49,482

According to the U.S. Bureau of Labor Statistics Average Hourly Earnings, the average cost to the respondent is $3.36.

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

a. There is no capital, start-up, operation, or maintenance costs.

b. Cost estimates are not expected to vary widely. The only cost is that for the time of the respondent (average of 5 minutes per respondent for phone study and 12 minutes for eSurvey).

c. There are no anticipated capital start-up cost components or requests to provide information.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government is estimated at $215,746.35. Table 3 below presents the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to the Contractor and will be included in the Contractor’s total cost.

TABLE 3: Estimated Cost to the Federal Government
Cost Item	Hours	Cost
VA-labor	560	$24,517
Contractor costs	-	$191,229.35
TOTAL		$215,746.35

The VA labor cost was estimated using a composite average salary and benefits figure of $43.78 per hour.¹ The amount paid to the Contractor for the survey effort includes as its major components the survey of Appellants who have had a decision on their appeal for total cost of $191,229.35.² These costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for administration, administration of the instrument, validation, data processing, providing a clean data file, project management and analysis, and reporting.

15. Explain the reason for any changes reported in Items 13 or 14 above.

There is no change.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Based on previous experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.

16A: TABULATION AND PUBLICATION

• Scan Survey and Monitor Response Rates

During the survey field-period, the survey will be scanned as they are received to ensure the accuracy of the name and telephone number data file. During the scanning process, the Board’s Contractor will inspect and remove duplicate surveys in case any individuals inadvertently received a second telephone call after completing a telephone survey.

The Contractor will host a password-protected Web site that will provide response rates from the Web-based surveys for the phone and online satisfaction surveys. Throughout the period that the survey is being fielded, data will be updated at a minimum on a quarterly basis. The Board’s leadership will be provided with passwords to access the site at any time. The Board will review the response rates on a quarterly basis and generate ideas to increase the response rates.

• Clean and Analyze Survey Data

Each quarter when the interviews have been completed, a raw ASCII data file will be produced, and the process of creating SPSS data files will begin. A SAS syntax program will be run to convert the ASCII data into separate SPSS data files representing quarter-to-date information from the phone and online survey. The “raw” SPSS data file will be saved into its own sub-directory (phone or online), and a copy of the original ASCII data file will be archived separately as a quality control measure. The contractor will analyze the SPSS data files, including conducting frequencies, cross-tabulations, and quadrant analyses. The analyses will be geared toward providing the Board’s leadership with user-oriented results.

• Create Quarterly Sample Disposition Reports

Quarterly disposition reports, which include total calls made, the number and percent of refusals, terminates, number of eSurveys deployed, number of completes, incidence, response, and cooperation rate reports on an electronic dashboard.

• Create Monthly Data Matrix

The Contractor shall provide the Board with monthly data matrices via a secure FTP site that is accessible to appropriate Board staff and management. Matrices shall provide cumulative summaries of all data month-to-date and year-to-date, on a secure FTP site, hosted by the Contractor.

16B: PROJECT TIMELINE

The major activities for the Board Appellant Satisfaction Survey project are structured by task, and are outlined below.

• Task 1: Conduct kickoff meeting and develop the Project Management Plan (PMP)

• Task 2: Contractor shall participate in face-to-face meetings with the Board’s leadership to review survey materials, sampling plans, and ongoing study details

• Task 3: Written Appellant Methodology Plan, Survey Samples and Sampling Plan

• Task 4: Design of two survey instruments (telephone survey instrument and online appellant eSurvey instrument)

• Task 6: Contractor completes 14,727 interviews (combination of eSurvey and telephone interviews)

• Task 7: Quarterly draft and final disposition reports for phone and eSurvey instruments

• Task 8: Compile and submit all quarterly and final reports, survey data files and statistical analysis

• Task 8.9: Monthly written progress reports

TIMELINe – call center satisfaction SURVEYS

Field Surveys	Continuous FY13-15
Analyze Data	Continuous FY13-15
Post Data to Reporting Site	Continuous FY13-15
Finalize Reports	Biannually FY13-15

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

Not applicable. (Not seeking such approval.)

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

1 FY 2009 Budget Estimates

2 This total is based on a cost estimate for the proposed contracted work.

- 8 -