Justification A 2900-0816 BVA Appellant

Supporting Statement A

2900-0816

Board of Veterans’ Appeals

Voice of the Veteran Appellant Satisfaction Survey

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection.

The mission of the Board of Veterans’ Appeals (Board) is to hold hearings and issue timely quality decisions on appellate matters.

Currently, the Board collects customer satisfaction data using the Customer Satisfaction Research Study, consisting of two separate survey instruments – the Appellant Satisfaction Telephone Survey and the Appellant Satisfaction eSurvey. The Board has collected customer satisfaction data with J.D. Power and Associates (Contractor) since FY 2014. The Board will continue to benefit from obtaining direct feedback from Veterans and other appellants regarding their experience with the Board’s appeals process. Specifically, the Veterans’ and other appellants’ feedback will provide the Board three key benefits: (1) Identify what is most important to them in determining their satisfaction with the Board’s appeals process; (2) determine how to improve their experience with the Board’s appeals process; and (3) serve to guide training and/or operational activities aimed at enhancing the quality of service provided to Veterans and other appellants.

The Board and the Contractor will continue to survey Veterans and other appellants who have had their appeal decided through the Board’s appeals process. This will enable the Board to gauge the effectiveness of its process delivering information and assistance to Veterans, as well as assess Veterans’ overall level of satisfaction with the Board’s appeals experience. In addition, the data will be used by the Board to make potential improvements to its operational processes and service delivery, which in turn, will enable the Board to serve Veterans and other appellants in the most efficient and effective way possible.

2. Indicate how, by whom, and for what purpose the information is to be used; indicate actual use the agency has made of the information received from current collection.

The Board has used the information collected from this survey to improve the information it provides to Veterans throughout the appeals process and to enhance the quality of appeals decisions it issues. The Board will continue to use the results of the survey to understand and quantify satisfaction levels of Veterans and other appellants who have gone through the benefit appeal process. These results will offer the Board critical inputs that can be used to assist decision makers to formulate operational changes in the Board’s appeals process to ensure that Veterans and other appellants are effectively served.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The Appellant Satisfaction Survey is composed of two separate survey instruments, a telephone survey and an online electronic survey (eSurvey). The telephone survey instrument is a brief assessment of appellant satisfaction with the appeals process and is tailored to collect email addresses for the more-in-depth online survey. Email invitations will be sent weekly to those appellants who provide an email address during telephone interviews. Respondents will be contacted via email no more than twice during a two-week period. Respondents will receive an initial invitation followed by a reminder email if they do not complete the eSurvey within one week. Respondents who did not provide an email address (i.e., opted out) will not be contacted for the eSurvey. The eSurvey instrument will measure various processes an appellant undertakes with the Board when appealing a decision. Data collection will occur during quarterly time frames.

The Contractor will conduct the initial telephone interview among Veterans and other appellants who have completed the Board’s appellant process. Using sample files provided by the Board, the Contractor shall randomly select an adequate sample to complete approximately 14,727 interviews (combination of telephone interviews and eSurveys).

Telephone calls will be made between 11 a.m. and 4 p.m. on Saturdays (no telephone calls on Sunday) and between 2 p.m. and 7 p.m. on weekdays in the potential respondent’s local time zone. Telephone calls will not be made on designated holidays. A maximum of seven (7) call attempts will be made to each potential respondent. Interviews will be conducted by live interviewers who will inform potential respondents who they are and why they are calling at the beginning of each call.

In conjunction with the Contractor, the Board’s staff will develop, administer, and analyze the survey data. Also, the Contractor in collaboration with the Board will select the telephone survey provider(s) that will maintain effective security and privacy procedures (e.g., data encryption).

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

Currently, the Board collects customer satisfaction information regarding appellants’ satisfaction with the appeals process by surveying Veterans and other appellants who have received an appeals decision from the Board. Specifically, the Board provides a sample to the Contractor on a monthly basis of all individuals who have been issued a decision in the previous month. The Contractor contacts individuals to participate in a 5-minute telephone survey and are asked at the end of the telephone survey to provide an email address to participate in a longer eSurvey online. If respondents agree to provide their email address, the Contractor sends an email invitation with the eSurvey link. Survey results are aggregated and included in quarterly results reports to the Board. The Appellant Satisfaction Survey instruments are designed as to not duplicate questions that are being addressed in the Board’s Hearing Experience Survey. Additionally, review of available data suggests that there are no outside sources of data which the Board could use to obtain a representative sample of appellants’ feedback on satisfaction with the appellant process.

To prevent duplication within the survey (i.e., calling the same appellant more than once) the Board and the Contractor will de-dupe the call list so that an appellant is only contacted after their most recent appeal has been adjudicated. Appellants with the same telephone number will not be contacted more than once in a six-month time period.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If the Board is unable to proceed with the collection of the data, it will not have the benefit of receiving feedback on what is important to appellants or how best to improve service for them and to best serve the needs of our Veterans. The collection of the data will enable the Board to continue to track and document improvements or declines in the appellant process over time.

The design and administration of the Appellant Satisfaction Survey incorporates significant measures to minimize burden on respondents (see Item 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and will produce valid and reliable results that can be generalized to the universe of study.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

A 60-day Federal Register Notice published at Vol. 82 FR 78, April 25, 2017, page 19140. No comments were received in response to the notice.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift shall be provided to respondents.

10. Describe any assurance of privacy, to the extent permitted by law, provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

An assurance of strict privacy, to the extent permitted by law, is made in the introduction respondents receive with the live telephone interviewer. Respondents are assured that answers given will be kept private to the extent of the law and will be used for research purposes only. Respondents will also be given the opportunity to opt-out of completing the survey during the introduction. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code). The Privacy notification is included in a list of frequently asked questions that respondents have. During the interview, J.D. Power responds as follows:

Question: Will my responses during this interview remain confidential?

Answer: Yes, your responses during this interview will remain private, to the extent permitted by law and will not affect your eligibility for current or future benefits. The answers you provide will not be linked to your name or contact information.

11. Provide additional justification for any questions of a sensitive nature (Information that, with a reasonable degree of medical certainty, is likely to have a serious adverse effect on an individual's mental or physical health if revealed to him or her), such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The survey instrument does not contain any questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information.

TABLE 1: ESTIMATED ANNUALIZED TIME BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Number of responses per respondent	Average burden per response (in hours)	Total burden hours
Appellants who have received a decision from the Board (telephone survey -80%)	9,753	1	.08	780
Appellants who have received a decision from the Board (eSurvey -20%)	1,595	1	.20	319
Total Hours				1,099

The U.S. Bureau of Labor Statistics (BLS) gathers information on full-time wage and salary workers. To derive average costs, we used data from the BLS’s latest available Current Population Survey, published in February 2017. According to that survey, the median weekly earnings of full-time wage and salary workers are $832.00. Assuming a forty (40) hour work week, the median hourly wage is $20.80. The following table presents the median hourly wage, the cost of fringe benefits (calculated at 100 percent of salary), and the adjusted hourly wage.

TABLE 2: ESTIMATED MONETARY BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Median hourly wage1	Fringe benefit	Adjusted hourly wage	Average time burden per response (hr.)	Total cost to all respondents
Appellants who have received a decision from the Board (telephone survey -80%)	9,753	$20.80	$20.80	$41.60	.08	$32,457.98
Appellants who have received a decision from the Board (eSurvey -20%)	1,595	$20.80	$20.80	$41.60	.20	$13,270.40
Total Cost						$45,728.38

According to the BLS Average Hourly Earnings, the cost to the respondent is $3.33 for the telephone survey, and $8.32 for the eSurvey. The estimated total cost to all respondents to be $45,728.38 (1,099 burden hours x $41.60 per hour).

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

a. There are no capital, start-up, operation, or maintenance costs.

b. Cost estimates are not expected to vary widely. The only cost is that for the time of the respondent (average of 5 minutes per respondent for the telephone study and 12 minutes for the eSurvey).

c. There are no anticipated capital start-up cost components or requests to provide information.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government is estimated at $244,768.74. Table 3 below presents the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to the Contractor and will be included in the Contractor’s total cost.

TABLE 3: Estimated Cost to the Federal Government
Cost Item	Hours	Cost
VA-labor	416	$28,936.96
Contractor costs	-	$215,831.78
TOTAL		$244,768.74

The VA labor cost was estimated using a composite average salary and benefits figure of $69.56 per hour.² The amount paid to the Contractor for the survey effort includes as its major components the survey of appellants who have had a decision on their appeal for total cost of $244,768.74.³ These costs include development/revision of the instruments, development/revision of the sampling plan, review of the instruments, locating respondents, programming the questionnaire for administration, administration of the instruments, validation, data processing, providing clean data files, project management and analysis, and reporting and briefings.

15. Explain the reason for any changes reported in Items 13 or 14 above.

The estimated number of respondents decreased by 3,379, from 14,727 to 11,348. This decrease is due to the Contractor adjusting its estimate based on response rates from the previous year. As the estimated number of respondents decreased, the estimated annual time burden also decreased accordingly, from 1,571 hours to 1,099. The total cost to all respondents has decreased from $49,482 to $45,728.38, due to two factors. First, the estimated number of respondents decreased, as noted. Second, the most recent data from the U.S. Bureau of Labor Statistics Average Hourly Earnings indicates an increase in average hourly earnings compared to previous years.

The estimated cost to the Federal Government has increased, from $215,746.35 to $244,768.74. Although estimated VA labor hours decreased, the estimated hourly composite average salary and benefits figure increased based upon current salary data. Finally, the Contractor costs increased based on a cost estimate for the proposed contracted work based on FY 2017 Contractor costs.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Based on previous experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.

16A: TABULATION AND PUBLICATION

• Scan Survey and Monitor Response Rates

During the survey field-period, the survey will be scanned as they are received to ensure the accuracy of the name and telephone number data file. During the scanning process, the Contractor will inspect and remove duplicate surveys in case any individuals inadvertently received a second telephone call after completing a telephone survey.

The Contractor will host a password-protected website that will provide response rates from the web-based surveys for the telephone and online satisfaction surveys. Throughout the period that the survey is being fielded, data will be updated at a minimum on a quarterly basis. The Board’s leadership will be provided with passwords to access the site at any time. The Board will review the response rates on a quarterly basis and generate ideas to increase the response rates.

• Clean and Analyze Survey Data

Each quarter when the interviews have been completed, a raw ASCII data file will be produced, and the process of creating SPSS data files will begin. An SAS syntax program will be run to convert the ASCII data into separate SPSS data files representing quarter-to-date information from the telephone and online survey. The “raw” SPSS data file will be saved into its own sub-directory (telephone or online), and a copy of the original ASCII data file will be archived separately as a quality control measure. The Contractor will analyze the SPSS data files, including conducting frequencies, cross-tabulations, and quadrant analyses. The analyses will be geared toward providing the Board’s leadership with user-oriented results.

• Create Quarterly Sample Disposition Reports

Quarterly disposition reports, which include total calls made, the number and percent of refusals, terminates, number of eSurveys deployed, number of completes, incidence, response, and cooperation rate reports will be provided by the Contractor via an electronic dashboard.

• Create Quarterly Data Matrix

The Contractor shall provide the Board with quarterly data matrices via a secure FTP site that is accessible to appropriate Board staff and management. Matrices shall provide cumulative summaries of all data quarter-to-date and year-to-date, on a secure FTP site, hosted by the Contractor.

16B: PROJECT TIMELINE

The major activities for the Board’s Appellant Satisfaction Survey project are structured by task, and are outlined below.

• Task 1: Develop the Project Management Plan (PMP) and conduct a kick-off meeting

• Task 2: Contractor shall participate in up to ten face-to-face meetings with the Board’s staff to review survey materials, sampling plans, and ongoing study details

• Task 3: Written Appellant Methodology Plan, Survey Samples, and Sampling Plan

• Task 4: Design two survey instruments (telephone survey instrument and online eSurvey instrument)

• Task 6: Contractor completes 14,727 interviews (combination of eSurvey and telephone interviews)

• Task 7: Quarterly and final disposition reports for telephone and eSurvey instruments

• Task 8: Compile and submit all quarterly and final reports, survey data files, and statistical analysis

• Task 8.9: Monthly written progress reports

TIMELINe – APPELLANT satisfaction SURVEY

Field Surveys	Continuous FY17-18
Analyze Data	Continuous FY17-18
Finalize Reports	Bi-annually FY17-18

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

Not applicable. (Not seeking such approval.)

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

1 This figure is based on Labor Force Statistics from the BLS Current Population Survey, last updated on February 8, 2017, and available at http://www.bls.gov/cps/cpsaat37.htm.

2 This composite figure was calculated using the average salaries and benefits of VA employees who will be working on the Appellant Satisfaction Survey.

3 This total is based on a cost estimate for the proposed contracted work based on FY 2017 Contractor costs.

- 8 -