0038 Just 2010 Final Part B

SUPPORTING STATEMENT
UNITED STATES PATENT AND TRADEMARK OFFICE
Customer Input – Patent and Trademark Customer Surveys
OMB Control Number 0651-0038
(April 2010)

B.

COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.

Universe and Respondent Selection

Random sampling is the primary method used when conducting the various types of
surveys. The primary groups surveyed include corporations/businesses, law firms,
educational institutions, and independent inventors. Computerized databases of these
major groups of USPTO customers are maintained and are used for random selection.
The USPTO pulls this information from the Patent Application Locating and Monitoring
(PALM) system and the Trademark Reporting and Monitoring (TRAM) system.
The PALM and TRAM systems keep detailed information on transactions conducted by
customers. Surveys are always geared towards a specific transaction and time frame.
Once the two criteria are established, all records in the PALM and TRAM systems that
meet the specific criteria are extracted to identify the universe. The transactions are
limited to those that deal with a particular organization or function within the USPTO,
e.g., an appeal, a petition, or other office-specific transaction. The time frame from
which the universe is selected is dependent on the expected universe size, the
frequency of the data collection effort, and time sensitivity of the questions to be asked.
With most of the customer surveys, the time frame usually includes transactions that
occurred within the previous 12 months. The universe can be as small as 1,000 records
for a 12-month period. Historically, 12-month periods have proved sufficient for
decision-making purposes. Longer periods introduce memory biases and overlapping
with previous collection efforts. Shorter time reference periods are desired when the
customer surveys are focused on time-sensitive issues, such as a recent change in
procedure or policy.
Since the majority of customer surveys use a customer, and not the transaction, as a
sampling unit, the initial file extracted from PALM and/or TRAM may contain a single
customer that is associated with over 100 transactions. When customers are the
desired sampling unit, file-cleaning procedures are performed to collapse duplicate
records and generate a single record for each individual at an establishment. When
records are collapsed, the number of total transactions that customer accounts for are
maintained. This value is then used for stratifying the sample and/or weighting the final
data set when desired.
Once the universe has been prepared, records are randomly selected by computer
program to obtain the desired sample size. Sample sizes are dependent on expected
response rates, desired statistical confidence and analyses to be performed, and

budgetary constraints. As a general rule of thumb, and given historic response rates,
goal sample sizes for USPTO customer surveys are 1,000 units for every level of
reporting desired. For an office-specific survey, where there is no need for further
breakdown of data, a sample size of 1,000 is used as the starting point, with budgetary
and specific analytical needs dictating deviations.
Response rates for voluntary customer surveys are usually between 25 and 50 percent.
This rate is typical for USPTO customer surveys and is considered adequate for making
product and service improvement decisions. In general, it provides the agency with 300
to 500 records to analyze.
2.

Procedures for Collecting Information

Although the USPTO is requesting that this generic clearance cover telephone and
customer surveys and questionnaires, the USPTO believes that the majority of the
surveys that will be conducted during the next three years will either be questionnaires
or customer surveys. But since the USPTO cannot predict with certainty exactly which
or how many of these surveys will ultimately be conducted, the agency is requesting an
extension of this generic clearance to cover all of these customer satisfaction surveys.
As part of the generic clearance guidelines, the USPTO is including general
descriptions and examples of the proposed survey vehicles. When the USPTO
determines that a survey is necessary and decides on the specific type of survey to be
used, the specific plans, methodology, scripts and copies of the survey will be provided
to OMB for review and approval under the generic clearance before the survey is
conducted.
•

Telephone surveys. The USPTO calls the respondent and either surveys the
respondent or schedules an appointment and faxes the survey questions to the
respondent. In addition, a script is prepared for the actual telephone interview so
that each telephone survey is conducted in the same manner.

•

Questionnaires and Customer surveys. The USPTO envisions that these surveys
will be conducted in both a paper and electronic format. When the surveys are
available in paper, they can be mailed to respondents with a cover letter or they can
be handed out to users of the USPTO’s various services or attendees at various
conferences.
In the case of the mail surveys, the USPTO will mail the surveys with a cover letter.
The cover letters generally explain why the survey is being conducted and request
that the customer take the time to complete the survey because the data gathered
from the surveys will be used for future planning and enhancements to current
initiatives. Additionally, the cover letter may provide mailing instructions, such as
using the self-addressed and stamped envelope included with the survey to return it
to the USPTO. In general, the USPTO follows-up on non-responses by mailing
reminders to the respondents and through phone contact. Mail surveys allow the
USPTO to ask questions about different areas, such as customer perceptions and

2

customer satisfaction with the services and processes. The surveys are primarily
designed in a multiple-choice format, with space provided for any additional
comments that the customer may wish to add.
Questionnaires and customer surveys are also used to survey users of the USPTO’s
various services or to survey attendees at various conferences. In these cases, the
survey forms are either handed to the respondents by the staff or left for attendees
to pick up as they enter or exit from various functions. If the completed surveys are
not handed directly back to a staff member, the respondents are instructed to drop
off their surveys or mail them back to the USPTO. These questionnaires are a mix
of fill-in-the-blank, multiple choice, or check boxes and also provide the customer
with the opportunity to provide comments.
In many cases, the USPTO will offer respondents the option to respond to a mail
survey through an electronic, web-based option. In some cases, the USPTO may
conduct a totally electronic survey, although the agency does not foresee electronic,
web-based surveys being used as the sole method for data collection, since many
customers have expressed a desire for the surveys to be available in a paper format.
The USPTO expects that the electronic, web-based surveys will always be used in
conjunction with the paper-based surveys.
When an electronic, web-based option is also available for a paper survey, the webbased survey will mirror the paper survey and will be hosted on the Internet. A URL
and instructions (including password and user IDs) for completing the web-based
survey will be included in the correspondence that accompanies the paper-based
surveys. When needed, the electronic survey instrument will be hosted on a secure
server. In the few instances where web-based surveys would be the primary data
collection method, customers will receive a survey announcement via an e-mail
message with a direct link to the web-based instrument. The e-mail will include
instructions for accessing and completing the web-based survey and will also
contain information on obtaining a paper-based survey and postage-paid return
envelope from the USPTO.
The USPTO envisions conducting both probability and non-probability surveys. For
mail and electronic surveys where the universe can be finitely defined, stratified random
samples will be selected using a computer program. Depending upon the analysis and
weighting needs, stratification will be based on the frequency of usage (how many
transactions they conducted in the given time period), affiliation (law firm, educational
institution, independent inventor, etc), USPTO-related function (technology center,
discipline, etc.) or any combination of these factors. Based on the review of the
extracted universe, strata will be defined to provide sufficient cases for meaningful
comparison among groups or group-level estimates. Once the universe is stratified, the
random sample generator will be applied, based on total desired sample size, to provide
a representative sample. Since response rates typically vary among stratum, data
weighting will take place in the analysis phase to maintain representativeness. Optimal
sample allocation techniques may be employed if the variance within strata (based on

3

previous data collection efforts) dictate that simple random sampling is insufficient to
produce strata-level estimates. Again, weighting would be applied in the analysis phase
to ensure population representation.
Sample sizes will be dependent on expected response rates, resource constraints, and
desired statistical confidence. When estimates of the entire customer population are
desired, samples will be selected to provide error margins of +/-5% at 95% confidence.
Confidence levels may be lowered to 90% if 95% appears prohibitive. Furthermore, if
group comparisons, trend analysis, and/or models are required, sample sizes will be
constructed to provide sufficient cases per variable or stratum. Significance testing will
be performed at the p=0.05 level and data variance (estimated from previous surveys)
will be used to ensure sample sizes are sufficient.
Non-probability surveys cover data efforts where respondents self-select themselves to
participate. The USPTO would simply offer a data collection instrument and let those
who choose to participate do so. Surveys placed in common areas are data collection
efforts where the USPTO would employ non-parametric analyses, but not make
inferences to the entire USPTO customer population. Specific sampling techniques
would not be used, and enumeration would remain open until sufficient responses for
meaningful indications were obtained.
For probability surveys where estimates can be made for the population as a whole,
base weights will be constructed at the time of sample selection. The base weight will
simply be the inverse of the sampling rate for each stratum. The final weight for each
responding customer is the product of the base weight times a nonresponse
adjustment. The nonresponse adjustment will take into account the response rate for
each stratum. The final weights will be incorporated into the data set and used in
relevant analyses to provide unbiased estimates for stratum-level estimates and the
selected customer population as a whole.
3.

Methods to Maximize Responses

In an effort to enhance response rates, the various types of surveys are tested or
reviewed to ensure that questions and instructions are clear, relevant, and
unambiguous. Sending an additional survey is the most common method for
maximizing responses in surveys employing follow-up procedures. Furthermore, for
surveys where the USPTO expects at least 100 responses, customers will be provided
the option of completing the survey via paper and mail or via a web-based survey on the
Internet. All survey correspondence will identify a USPTO employee that the customer
can contact for questions or a specific survey need.
All sample surveys are expected to generate sufficient response that will produce
reliable data that can be generalized to the universe studied.

4

4.

Testing of Procedures

The various types of surveys (and any associated instructions) are reviewed by USPTO
employees who use or are familiar with the product or service under study. In some
cases, the USPTO may pretest the various types of surveys with up to nine external
customers. The surveys are tested the first time that they are conducted.
5.

Contact for Statistical Aspects and Data Collection

The Office of Patent Quality Assurance of the USPTO is responsible for coordinating
and administering the customer surveys and related data collection efforts. Paula
Hutzell, Director, Office of Patent Quality Assurance, can be reached via phone at 571272-0531. Martin Rater is the agency resource for customer surveys and statistical
projects, and can be reached via phone at 571-272-5966 or via e-mail at
martin.rater@uspto.gov.

5