Proposed Survey Enhancements

Supplier Perception Survey Response Analysis Enhancements

About the Supplier Perception Survey

The Supplier Perception Survey is a component of the U.S. Department of Veterans Affairs (VA) Office of Acquisition & Logistics (OAL) Supplier Relationship Transformation (SRT) initiative. This survey is one of many enterprise feedback initiatives, through which Ambit collects quantitative and qualitative information, to holistically assess VA Acquisitions Supply Chain Management policies and practices.

Current Response Analysis Practices and Limitations

For the most recent iteration of the Supplier Perception Survey, chi-square goodness of fit tests were performed on key supplier attributes to assess representativeness of respondents. However, this analysis was limited in that the analytics team did not have access to data sources containing ‘absolute’ and objective supplier information in conducting this assessment. Instead, supplier self-reported attributes (e.g., business category) were compared with self-reported attributes from the baseline Supplier Perception Survey conducted in 2010. This assessment – while admittedly not ideal or optimal – was essentially the best option among a limited number of sub-optimal options.

Future Enhancements to Response Analysis

Ambit is currently engaged in a multi-pronged effort to gain access to data sources that will allow for more technically sound supplier survey response analysis for subsequent surveys. Table 1.1 below lists the data sources that have been identified as potential sources for future response analysis.

Table 1.1 Data Sources Identified for Use in Supplier Perception Survey Response Analysis

Data Source	Data Source Owner	Current Status
VA Electronic Contract Management System (eCMS)	Veterans Affairs	Ambit has held discussions with the VA eCMS team to discuss additional supplier fields available through eCMS.
Central Contractor Registration Database (CCR)	General Services Administration	Ambit is actively developing extract, transform, load (ETL) procedures for CCR database.
Federal Procurement Data System	General Services Administration	Ambit in the beginning phases of investigating available data elements and data quality of the FPDS data source.
Commercial List and Append Services	Various Commercial Vendors	Ambit has received a data dictionary of fields available through Hoovers corporate info data sources.

Between the 4 distinct data sources currently being investigated, Ambit is confident that we will be able to apply a far more rigorous approach to responder analysis for the next iteration of the Supplier Perception Survey.

The remainder of this document outlines key milestones as well as targeted completion dates in our assessment of each of these data sources.

VA Electronic Contract Management System

• Ambit to provide the eCMS team with a data specifications document that outlines the supplemental supplier information being requested (ETA: 5/4/2012)

• VA eCMS team to provide Ambit with a data extract that can be used to assess field ‘coverage’ and quality using the sampling frame from the Q2 FY 2012 survey (ETA: 5/15/2012)

• Ambit to complete analysis on data quality of fields provided by eCMS team (ETA: 5/18/2012)

Central Contractor Registration Database (CCR)¹

• Ambit to request assistance of OMB to identify a point of contact with GSA that can facilitate Ambit gaining greater understanding of CCR data (ETA: 5/2/2012)

• Ambit to engage GSA on CCR data (ETA: 5/16/2012)

• Ambit to complete coding development for CCR dataset processing (ETA 5/25/2012)

• Ambit to complete analysis on incremental value CCR data may provide over eCMS data (ETA 6/1/2012)

Federal Procurement Data System (FPDS)²

• Ambit to request assistance of OMB to identify a point of contact with GSA that can facilitate Ambit gaining greater understanding of FPDS data (ETA: 5/2/2012)

• Ambit to engage GSA on FPDS data (ETA: 5/16/2012)

• Ambit to complete coding development for FPDS dataset processing (ETA 5/25/2012)

• Ambit to complete analysis on incremental value FPDS data may provide over eCMS data (ETA 6/1/2012)

Commercial Data Sources

• Ambit to submit a list of desired variables to commercial vendor (Hoovers) (ETA 5/4/2012)

• Ambit to determine whether pricing of commercial list meets budgetary requirements (ETA 5/8/2012)

• Ambit to conduct ‘proof of concept’ exercise using commercial list to determine if match rate on current sample of supplier roster meets expectations (ETA 5/15/2012)

By June 1, 2012, Ambit expects to have a final perspective on the data sources that should be leveraged for response analysis in future iterations of the Supplier Perception Survey. With the next iteration of the survey not scheduled to launch until July 2012, this leaves over a month for Ambit to procure all relevant data sources and structure those data sources for use in sampling frame development and analysis. It is expected that all data preparation will be final no later than June 29, 2012.

1 It is currently assumed that CCR and FPDS data sources are closely related, and can be addressed collectively along the same timeline.

2 It is currently assumed that CCR and FPDS data sources are closely related, and can be addressed collectively along the same timeline.

2