3206-0236 Customer Satisfaction Surveys Section B_rs_OPM_revX

Customer Satisfaction Surveys (OMB No. 3206-0236)

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Most Customer Satisfaction Surveys are completed by Federal employees, although the populations surveyed vary based on client needs. We conduct surveys on a reimbursable basis for customer agencies with a wide variety of needs, preferences and/or requirements. We strive to meet the needs of each of our customers in the most efficient and cost-effective manner and we always present the pros and cons of the various possible methodologies. That said, our customers’ needs and preferences vary greatly so we do not have fixed rules regarding sample size. Some of our customers, for example, prefer to conduct a census survey to allow their entire target population an opportunity to respond. Other customers may not need or want a census survey, or may lack the resources necessary to conduct a census, in which case we sample.

In every case, we determine whether to use a census or a sample based on the customer’s needs/requirements and the anticipated number of returns necessary to yield the desired margin of error, both for the entire sample and for relevant demographic groups. Many times the size of the demographic groups (e.g., customers of a particular site, users of a particular service) is not known a priori, in which case it may be difficult or impossible to generate an accurate sampling plan. Other times, desired breakouts are known to have small or large n’s, in which case we can, and have, used sampling and weighting. However, the cost to the government of sampling must be weighed against the cost of conducting a census, and must be considered in light of other factors such as the desire to provide potential respondents a voice, the complexity of anticipated statistical analyses, the desire not to over-survey, requirements of the project timeline, and other requirements (e.g., legal) and desires (e.g., to appear totally transparent) of the client. For the particular example of contractors noted in your question, sampling would yield the desired margin of error in the overall sample, but would not yield the desired margin of error in various breakout groups. In the past we did sample for the survey, using various methods, until the cost of doing so surpassed the cost of conducting a census. Because our sample vs. census decisions are responsive to our customers’ needs and requirements, and we do not have fixed sampling rules, if the client’s needs or the characteristics of the population were to change we would reevaluate our methodology accordingly.

The average response rate is approximately 20%. The sampling methods, and whether to sample at all, are based on a combination of factors, including the expected number of returns needed to yield an overall margin of error of +/- 2.5 percentage points (assuming a conservative score of 50%), the number needed to maximize the margin of error of key breakout groups, client requests, and available customer data.

2. Describe the procedures for the collection of information including:

We are not familiar with this terminology, but assume the question relates to whether we consider the breakout groups for which we will produce results (i.e., overall vs. demographic subgroups). As described above, we do consider the sample size needed to yield the desired margin of error in both the overall sample and for key breakout groups. Also as described above, many factors are considered so margin of error is not the sole determinant in our decision process. Due to the diversity of our clients and their customers, each survey scenario is evaluated individually to determine if sampling or a census will yield the most useful information in the most cost-effective way.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Survey recipients are generally sent at least three reminders and collected data is checked against population figures (when known) to ensure representatives across customer groups. Client agencies are asked to inform their customers of the survey and to encourage their participation. Notification and reminder notices include, when possible, examples of improvements made based on previous customer satisfaction surveys.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Any pilot testing of new survey items or methods is conducted by Federal Government employees or groups of fewer than 10 non-Federal Government employees.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Steven R. Burnkrant, Ph.D.

Steve.Burnkrant@opm.gov

202-553-1279

Renée Vincent, M.S.

Renee.Vincent@opm.gov

202-553-1345

Bernard J. Nickels, Ph.D.

Bernard.Nickels@opm.gov

202-553-1224