Supporting Statement B

Customer Satisfaction Surveys

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The population of potential respondents varies by client/survey/project, but may include military contractors, users of Government-produced products and services, and job applicants. As such, the universe of potential respondents is large but unknown and dependent on a particular survey. For a particular survey, the population can vary from year to year, depending on how many people have received products and services. In a given year, the total number of potential respondents is estimated at 100,000. The average response rate of past administrations has been 30%, which also serves as an estimate for future administrations.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

Sampling methodology varies by project, characteristics of the customer population, and our client’s needs. The need for sampling is determined by multiplying the population size by the expected delivery rate (approximately 90%) and response rate (30%) and comparing the result to the sample size needed to obtain a margin of error of ± 2.5 percentage points. If the required sample size is 75% or more of the estimated actual sample size (to account for lower than expected delivery or response rates), then no sampling is used. If sampling is used, stratification and other procedures will be designed to best suit the characteristics of the population and the needs of our client. Simply random sampling is most common, although other methods may also be used, such as top-down selection based on such factors as the customer’s importance to our client and the degree of interaction with our client.

• Estimation procedure,

The need for sampling is determined by multiplying the population size by the expected delivery rate (approximately 90%) and response rate (30%) and comparing the result to the sample size needed to obtain a margin of error of ± 2.5 percentage points. The margin of error is calculated using the standard binomial formula, with p = .50 and the Finite Population Correction. If the required sample size is 75% or more of the estimated actual sample size (to account for lower than expected delivery or response rates), then no sampling is used. If sampling is used, the sample size is that which will give a margin of error of ± 2.5 percentage points after accounting for undeliverable notifications and non-response.

• Degree of accuracy needed for the purpose described in the justification,

The target margin of error is at most ± 2.5 percentage points.

• Unusual problems requiring specialized sampling procedures, and

If sampling is needed, then more important customers and those with more interaction with our client may be weighted more heavily in selection.

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

We recommend biannual administrations, although some clients require annual administrations for tracking scores in a Balanced Scorecard or similar matrix of organizational performance indicators.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

In addition to the initial survey notification, respondents receive at least two reminder notices. If the response rate is low after the second reminder, a third may be sent. When possible, clients publicize the survey in newsletters, on websites, etc. For surveys administered on-line, we make the survey as easy as possible to complete, by using direct-link access and save-and-return functionality. After data collection, respondent demographics are, whenever possible, compared to population demographics to ensure representativeness. If the data is found not to be representative, the results may be weighted. If certain customers have a greater chance of selection than others (e.g., those with daily interaction with our client vs. those who interact with our client only once a year), the results are treated as indicators of satisfaction among the “most important” customers, not as representative of every possible customer.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

Surveys are pilot tested with a small set of either actual customers or subject matter experts within our client agency, as appropriate. The pilot test serves to validate survey content and administration procedures, and to estimate the actual time needed to complete the survey.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

All survey administrations and analyses, including statistical aspects of the designs, will be led by project managers in the Assessment Service Branch of OPM. Project managers have either a Master’s or Doctoral degree in Industrial/Organizational Psychology or a closely related field. All project work is conducted under the guidance of one or more of the following senior psychologists:

Henry Thibodeaux, Ph.D, 202-606-7228

Steve Burnkrant, Ph.D., 202-345-7415

Kimberly Wells, Ph.D., 202-606-9088

Alexis Adams-Shorter, Ph.D., 202-606-1865

Tarl Kudrick, Ph.D., 202-606-2597

Jacqueline Brucker, Ph.D., 202-606-5055