201405-2900-001_VOV_Call Center_Justification A(CR 1 23 15) 04 09 15

Department of Veterans Affairs, Veterans Benefits Administration (VBA)

Voice of the Veteran (VOV) Call Center Satisfaction Survey

2900-0744

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The mission of the Veterans Benefits Administration (VBA) National Call Centers is to provide information and assistance to clients (e.g., Servicemembers, Veterans, beneficiaries, etc.) regarding the various programs, services, and benefits offered by VBA.

As part of VBA’s continuing commitment to improve the overall quality of service, VBA will benefit from obtaining direct feedback from its clients regarding their recent call center experience. Specifically, the clients’ feedback will provide VBA three key benefits: 1) identify what is most important to its clients in determining their satisfaction with their call center experience; 2) determine what to do to improve the call center experience and 3) serve to guide training and/or operational activities aimed at enhancing the quality of service provided to its clients.

VBA and the contractor will survey clients who have contacted one of seven National Call Centers, the Education Call Center in Muskogee and the National Pension Call Center in Philadelphia. Interviewing began in January 2010 and will continue through FY18.

This plan enables VBA to track and compare call center performance overall, as well as at each call center. This will further enable VBA to gauge the effectiveness of its call centers in delivering information and assistance to its clients, as well as assess the clients’ overall level of satisfaction with the call center experience. In addition, the data will be used by VBA to make improvements to the call center operational processes and service delivery, which in turn, will enable VBA to serve its clients in the most efficient and effective way possible.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from current collection.

Survey data is cascaded throughout VBA via an online reporting tool that is updated on a daily basis with survey returns collected the previous day, and through monthly information sessions. The monthly sessions include a review of call center performance in aggregate, as well as individual deep dives into call center performance for each center. This transparent reporting model allows Leadership and call centers to make actionable changes that will positively affect call satisfaction. The information collected from the surveys during FY10-FY14 enabled VBA to understand, quantify and compare call center satisfaction levels among its clients. The results offered VBA critical inputs to formulate operational changes in the call center environment and ensured that VBA’s clients are effectively served.

Based on the recommendations of the call center research program thus far, VBA has identified 100 service enhancements and have implemented 85 of those service enhancements to effect process improvements within the call centers (i.e., Virtual Hold and Scheduled Call Back technologies were implemented to establish greater convenience for clients, Public Contact Representatives (PCRs) Level II technical assessments, revised PCRs telephone call scripts, launched a SharePoint discussion board for the National Call Centers and training coordinators, established mandatory consistent 2 hour weekly development sessions across NCCs, ECC, and NPCC, hired and trained 250 PCRs to address increased call volume and wait times, Lexus Nexus upgrade to allow for remote proofing for eBenefits, etc.).

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

Information Technology is not feasible at this time. The contractor, J.D. Power (JDP), will conduct CATI (Computer Assisted Telephone Interview) surveys among Veterans who have recently contacted VBA’s National Call Centers, the National Pension Call Center, and the National Education Call Center. Calls will be made to respondents no sooner than 24-48 hours after the respondent speaks with a representative at the call center. Respondents will be randomly selected to participate from each of the six sample files provided by VBA throughout each month until a minimum of 55-56 interviews per call center are completed from each sample file. This will produce a total of 3,000 completes per month for a total of 36,000 interviews completed annually.

Telephone calls will be made between 11 a.m. and 4 p.m. on Saturdays (no telephone calls on Sunday) and between 2 p.m. and 7 p.m. on weekdays in the potential respondent’s local time zone. Telephone calls will not be made on designated holidays. A maximum of six call attempts will be made to each potential respondent. Interviews will be conducted by live interviewers who will inform potential respondents who they are and why they are calling at the beginning of each call.

In conjunction with the contractor, JDP, VBA staff will develop, administer, and analyze the survey data. Also, the contractor in collaboration with VBA, will select the telephone survey provider(s) that will maintain effective security and privacy procedures (e.g., data encryption).

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

VBA is currently not collecting any other call center satisfaction data from clients, so there is no duplication across other efforts conducted within the VBA. Additionally, review of available data suggests that there are no outside sources of data which the VBA could use to obtain a representative sample of the clients feedback on call client satisfaction.

To prevent duplication within the survey (i.e. calling the same client more than once) VBA and the contractor will de-dupe the call list so that a client is only contacted for their most recent call center experience regardless of the number of calls made to a National Call Center for two or more different reasons. Callers using the same telephone number will not be contacted more than one time in a six month time period.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If VBA is unable to proceed with the collection of the data, it will not have the benefit of continuing to receive client feedback on what is important to them or how best to improve their service and to best serve the needs of our clients. The collection of the data will enable VBA to track and document improvements or declines in call center client satisfaction and service delivery over time.

The design and administration of the Call Center Satisfaction Survey incorporates significant measures to minimize burden on respondents (see section 12). There are currently no technical or legal obstacles to reducing burden using the planned methods.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no special circumstances that would require respondents to prepare or submit the documents outlined above, or respond in fewer than 30 days. The surveys will be designed and carried out with appropriate scientific rigor, and will produce valid and reliable results that can be generalized to the universe of study.

8. Part A: If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

The 60-day Federal Register (FR) notice was published on July 11, 2014, Volume 79, No. 133, page 40205. No comments were received.

8. Part B: Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and record keeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances, which preclude consultation every three years with representatives of those from whom information is to be obtained.

VBA’s contractor for this survey is J.D. Power. VBA is consulting with J.D. Power regarding their Call Center Certification Program, which benchmarks call

center performance across a number of industries within the private sector. The J.D. Power Call Center Certification program is an event-driven approach to measuring satisfaction. That is, for each call center that is benchmarked, they survey clients who have interacted with that center. Recency within the J.D. Power methodology is defined as one to five day dialing period post call to the call center. J.D. Power chose this time frame after conducting empirical studies aimed at determining which of two methods (immediate or delayed surveying) yields the most valid/reliable results. Those studies indicated that immediate transfers overestimate satisfaction scores and yielded less reliable and less valid results than those obtained from the slight delayed surveying approach.

The ability to leverage J.D. Power methodology and benchmark data offers VBA a set of well-established and accepted industry performance benchmarks across numerous industries from which to compare and contrast various reporting metrics.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift shall be provided to respondents.

10. Describe any assurance of privacy to the extent permitted by law, provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

Respondents are assured that answers given will be kept private to the extent of the law and will be used for research purposes only. The information that respondents supply is protected by law (the Privacy Act of 1974, 5 U.S.C. 522a and section 5701 of Title 38 of the United States Code). Respondents will also be given the opportunity to opt-out of completing the survey during the introduction.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The survey instrument does not contain any questions of a sensitive nature.

12. Estimate of the hour burden of the collection of information.

TABLE 1: ESTIMATED ANNUALIZED TIME BURDEN, BY RESPONDDENT GROUP
Population surveyed	Number of respondents	Number of responses per respondent	Average burden per response (in hours)	Total burden hours
Veterans who have contacted a call center	36,000	1	.10	3,600
				3,600

TABLE 2: ESTIMATED MONETARY BURDEN, BY RESPONDENT GROUP
Population surveyed	Number of respondents	Estimate hourly wage	Average burden per response (hr.)	Total cost to all respondents
Veterans who have contacted a call center	36,000	$24	.10	$86,400
	36,000			$86,400

According to the U.S. Bureau of Labor Statistics Average Hourly earnings, the cost to the respondent is $2.40. ¹

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

a. There is no capital, start-up, operation, or maintenance costs.

b. Cost estimates are not expected to vary widely. The only cost is that for the time of the respondent (average of 6 minutes per respondent).

c. There are no anticipated capital start-up cost components or requests to provide information.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The total cost to the Federal Government is estimated at $780,027.18. Table 3 below presents the labor and contracting costs for conducting the surveys. Operational costs will be outsourced to a contractor and will be included in the contractor’s total cost.

TABLE 3: Estimated Cost to the Federal Government
Cost Item	Hours	Cost
VA-Labor	600	29,763
Contractor Costs		750,264.18
TOTAL		$780,027.18

The VA Labor cost was estimated using a the base hourly wage of $47.87 for a GS 13 step 4 for 500 hours and the base hourly wage of $58.28 for a GS 14 step 5 for 100 hours.² The amount paid to the contractor for the survey effort includes as its major components the survey of veterans who have contacted a call center for total cost of $780,027.18.³ These costs include development of the instruments, development of the sampling plan, review of the instrument, locating of respondents, programming of the questionnaire for administration, administration of the instrument, validation, data processing, providing a clean data file, project management and analysis, and reporting.

15. Explain the reason for any changes reported in Items 13 or 14 above.

There are no changes in items 13 and 14. The expiration date is being added to the survey.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Based on previous experience, Section 16A below documents the proposed project plan for the tabulation and publication of survey data, including the analytical techniques and database management strategies to be employed. Following this, Section 16B provides an overview of the project timeline.

16A: TABULATION AND PUBLICATION

• Monitor Response Rates

J.D. Power (JDP) will monitor fielding daily to ensure that the monthly interviewing quota of 333-334 completes or the sampling file quota of 55-56 completes per sample file are met for each call center. If any issues arise that may prevent quota fulfillment, JDP will notify VBA and arrange for a make-up sample collection day to receive additional sample.

• Clean and analyze survey data.

Each month when the interviews have been completed, a raw ASCII data file will be produced, and the process of creating SPSS data files will begin. A SAS syntax program will be run to convert the ASCII data into separate SPSS data files representing month-to-date, fiscal year-to-date, and quarter-to-date information for each call center and the national data. Each call center’s “raw” SPSS data file will be saved into its own sub-directory, and a copy of the original ASCII data file will be archived separately as a quality control measure. The contractor will analyze the SPSS data files, including conducting frequencies and cross-tabulations. The analyses will be geared toward providing VBA staff with user-oriented results.

• Create monthly sample disposition reports.

Monthly disposition reports, which include total calls made, the number and percent of refusals, terminates, number of completes, incidence, response, and cooperation rate, will be provided to VBA at the end of each fielding month.

• Create monthly data matrices.

The contractor will provide VBA with monthly data matrices via a secure FTP site that is accessible to appropriate VBA staff and management. Matrices shall provide cumulative summaries of all data month-to-date and year-to-date, on a secure FTP site, hosted by the contractor. The reports shall provide data at the National and call center levels for month-to-date, quarter-to-date, and fiscal year-to-date aggregations.

• Conduct daily data loads to an online reporting tool.

JDP will load the call center data collected to online reporting tool (i.e. VBA’s Voice of the Veteran (VOV) Reporting Site) that is updated on a daily basis (Monday thru Friday, excluding holidays) with survey returns collected on the previous day. The online reporting tool will be accessible to VBA and display results aggregated at the National level as well as tiered by each individual call center. At the end of each month, data on the reporting tool will be finalized and weighted. JDP will notify VBA each month once the final month end data has been uploaded.

• Conduct biannual formal briefings.

The contractor will prepare semi-annual overviews of data collected for VBA leadership. Results are published within the Performance Accountability Report, and budget submissions. VBA is currently in the process of designing a web site to publicly display its customer service results as recommended in a GAO report dated October 2014. This overview will be delivered in-person to VBA leadership and outline data trends and recommendations based on the data collected. The contractor shall provide VBA with 25 color copies of the presentation as handouts for leadership at the meeting.

16B: PROJECT TIMELINE

The VBA Call Center satisfaction survey project major activities for the contractor are structured by task, and are outlined below.

• Task 1: Conduct kickoff meeting and develop the project management plan (PMP)

• Task 2: Participate in monthly meetings with VBA staff to review cooperation rates, disposition of sample, and ongoing study details

• Task 3: Written Call Center Methodology Plan, Survey Samples and Sampling Plan

• Task 4: Create monthly sample disposition reports

• Task 5: Host Toll-free telephone assistance line, internet-based help desk

• Task 6: Conduct daily data uploads to VOV Reporting Site

• Task 7: Create monthly data matrix

• Task 8: Conduct biannual formal briefings

TIMELINe – call center satisfaction SURVEYS

Field Surveys	Continuous FY15-18
Analyze Data	Continuous FY15-18
Post Data to Reporting Site	Continuous FY15-18
Finalize Reports	Biannually FY15-18

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

We are not seeking such approval, as this will be a telephone survey with no vehicle for respondents to complete without answering via the telephone.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

This submission does not contain any exceptions to the certification statement.

1 The median hourly wage of all occupations in the United States is estimated by the U.S. Bureau of Labor Statistics at $24.00.

2 Based on the 2015 GS Wage Table

3 This total is based on a cost estimate for the proposed contracted work.

- 9 -