Supporting Statement B_ISOO Cost Collection

SUPPORTING STATEMENT – PART B

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.  Description of the Activity

The National Industrial Security Program (NISP) was established by Executive

Order 12829, January 6, 1993. NISP policies and procedures are promulgated in the NISP Operating Manual (NISPOM). The Secretary of Defense is designated as executive agent for the NISP. The NISP extends to all federal departments and agencies which permit private sector facilities access to classified information via contract, license, grant or other authorized means. Executive Order 12829 required agency heads to account each year for intra-agency costs associated with the implementation of the NISP and to report these costs to the Director, Information Security Oversight Office (ISOO), which is responsible for including them in annual reports to the President.

In furtherance of this requirement, and pursuant with the reporting requirement

of 32 CFR Part 2001, Subpart F, section 2001.61(b); classified National Security Information; Final Rule, the Secretary of Defense, acting as executive agent for the NISP, is obligated to collect cost estimates for classification-related activities of contractors, licensees, certificate holders, and grantees and report them to the ISOO annually.

The target population of 3,888 cleared facilities, as categorized as AA, A, B, C, D, consist of active, cleared industry facilities under the National Industrial Security Program approved for classified storage. These categories are taken from the DSS Industrial Security Facility Database (ISFD) and reflect the size, scope, and complexity of a facility’s security program with AA being the largest and D being the smallest. The ISFD is maintained by the Defense Security Service (DSS) and is a repository of information about Department of Defense cleared contractor facilities. The ISFD has internal users and external users with limited access. DSS representatives update the repository daily. These facilities are stratified using multiple quantifiable identifiers to create a scale of facility complexity with facility labels of AA, A, B, C, and D.

Facility breakdown within the stratification is as follows:

AA. 21; A. 75; B. 120; C. 297; D. 3,375

For categories AA through C, a census survey is used to collect information on these facilities due to the small number of facilities per category and the simplicity of the survey questions. To reduce the public burden, and comply with guidelines set forth by the government-industry NISP Policy Advisory Committee, a sample survey is used for Category D due to its large number of facilities.

2.  Procedures for the Collection of Information

The facility points of contact, referred to as Facility Security Officers (FSO), will receive an email detailing their inclusion in the survey. The email contains a link to the two question survey. The FSO will complete the data collection using a survey vendor that will collect and provide results to DSS. FSOs are selected in order to minimize measurement error, as their primary function is to manage a facility’s security program, which provides them with access to information regarding security labor.

As described above, the population is prime for stratification. The collection is deployed to one central point of contact for a facility. Facilities are set into the stratum by multiple factors directly related to this collection effort. Since facility categories AA through C are relatively small, DSS plans to perform a census on the entire population. For category D, DSS randomly samples approximately 1/3 of that category’s population as done in past years. The five stratified groups are then as follows:

AA. 21; A. 75; B. 120; C. 297; D. 1,040

Participants will be randomly selected from stratum D using a random number generator. Based on last year’s results, the expected response rate to the collection ranges from approximately 70% to 80% for each stratum.

Once the data from the sample survey is collected, the estimated total

security cost for each facility will be calculated, and a 95% confidence interval will be estimated using the Student's t-distribution. The upper bound of the interval is used by ISOO to estimate the maximum cost for security labor amongst cleared, active facilities in the NISP.

ISFD maintains many data elements pertaining to a facility’s size, scope,

and complexity and by using that data DSS can further stratify most to least complex facilities within each stratum. Additionally, ISFD provides details which can be used to find responding facilities that are similar to non-responding facilities. From this, DSS will apply a K-nearest-neighbor approach to impute missing values based on neighboring data points of a similar nature.

When missing values are accounted for, DSS will calculate the mean and standard deviation for each stratum. After removing outlying data points, DSS will re-calculate the mean for each strata. Using the average per stratum and multiplying it out by the total amount of facilities within the stratum, DSS then has an accurate estimate for the total cost of the category. By taking the sum of the total estimated cost for each of the five stratum, DSS will have the total estimated cost for the population as a whole.

The survey data collected by DSS is not collected for the purpose of developing

model-based estimates of target population characteristics or projections of future values. DSS does not release and disseminate detailed responses for individual respondents to the public.

3.  Maximization of Response Rates, Non-response, and Reliability

DSS will pre-notify selected participants prior to deployment of the collection effort. During the data collection period, non-respondents will be sent e-mail reminders weekly to ensure timely completion of the collection.

The general methodology used to estimate security costs incurred by industry relate specifically to the protection of classified information as may be applicable to respondent companies. The methodology captures the most significant portion of industry’s costs, i.e., labor. Labor costs are also relatively simple to identify, whereas non-labor costs are difficult to define and compile. The respondents are requested to compile their annual security labor cost in dollars. The labor cost, when identified as an estimated percent of each company’s total security costs, enables the respondent’s to calculate their total security costs without having to undergo a time-consuming, costly, and difficult analysis on non-labor costs. A large portion of costs billed to security can be extracted from contractor budget systems already in place. Thus, the NISP is not imposing a new cost accounting structure on the contractor. The methodology was drafted and supported by joint government-industry NISP Policy Advisory Committee and was accepted for implementation by industry signatories.

As the collection itself only consists of two clear questions, the simplicity of the form will encourage a high response rate. ISFD is updated daily which decreases the likelihood of distributing the data collection to terminated facilities and respective points of contact. DSS plans to re-interview FSO that provide outlying responses to determine if they are correct security costs.

DSS maintains a high working relationship with the respondent field and has consistently experienced high response rates on data collection efforts. Approximately 80% of selected facilities participate in the survey. For categories AA through C, the response rate for each census ranged from 78% to 81%. However, some surveys are not completed correctly, which lowers the usable rate to between 68% and 72% of all facilities. For the sample survey of Category D, the response was 73%, which lowered to 66% when incomplete responses were removed. We expect similar response rates for future data collections.

For Category D, a random sample of approximately 1,100 facilities will be selected. Approximately 70% of Category D facilities have responded to past surveys. In years past, this has produced a margin of error within 10% of the sample mean, which has been an acceptable level of precision for ISOO.

The ISOO releases the estimated security expense in aggregate. DSS does not release calculations, statistics, or any other data specific to a facility or its category.

4.  Tests of Procedures

The process has been tested over 11 years of deployment of this action. Confidence intervals have been consistent with previous estimates. In consultation with the government-industry NISP Policy Advisory Committee and approval from ISOO, DSS plans to refine the sampling technique, specifically with regard to the precision for category D, to potentially improve utility and minimize burden.

5.  Statistical Consultation and Information Analysis

Primary POC: Preston Harper 571 - 305 - 6358 or

Chris Pirch 571 - 305 - 6241

Defense Security Service, Business Enterprise