OMB Clearance Request for the Global Intellectual Property Academy Program Survey

OMB Clearance Request
for the
Global Intellectual Property Academy Program Survey

A. Survey Background
The United States Patent and Trademark Office (USPTO) established the Global
Intellectual Property Academy (“GIPA” or “Academy”) in 2006 to offer training programs
on enforcement of intellectual property rights, patents, trademarks, and copyrights. The
Academy’s programs are designed to meet the specific needs of foreign government
officials concerning the various intellectual property topics. By attending these
programs, foreign government officials learn about global intellectual property protection
and enforcement and discuss strategies to handle the protection and enforcement
issues in their respective countries. In 2008, the Academy provided training to foreign
government officials from 127 countries on a variety of topics, such as IP protection,
enforcement, and technology transfer. It is through the Academy’s various programs
that the USPTO is instrumental in achieving its objectives of halting intellectual property
theft and advancing IPR policies.
B. Survey Purpose
The USPTO’s Office of Intellectual Property Policy and Enforcement (OIPPE) is
sponsoring this customer satisfaction survey of the international participants in the
United States-based 2008 Global Intellectual Property Academy programs. They have
contracted with the Federal Consulting Group “FCG”, which is part of the U.S.
Department of the Interior. The FCG is an independent evaluation organization which
provides consulting services to other Federal agencies. The survey itself will be hosted
by a third-party provider, Survey Monkey, which is frequently used by Federal agencies
to host their surveys. This survey is a one-time survey that will be used as a pilot for
future surveys that the USPTO plans to develop to evaluate the Academy’s programs.
The primary reason that the USPTO is conducting this pilot survey is to fulfill its
requirements for the U.S. Government’s accountability process. Under the Program
Assessment Rating Tool “or PART”, the USPTO is required to gather data concerning
several different performance measures and report the data to the Office of
Management and Budget (OMB) to be used in its FY2009 report. The USPTO also
plans to use some of the collected data in their fiscal year budget justifications.
The USPTO is also conducting this pilot survey in order to establish an ongoing survey
methodology of GIPA participants. For this pilot survey, the USPTO is surveying the
total number of English-speaking foreign officials who attended the 2008 United Statesbased training programs at the Academy. The USPTO will use the information gathered
and the experience gained from conducting the pilot GIPA alumni survey to develop
other surveys. The alumni survey was chosen as the pilot survey in part because its

questions concern many of the same topics that are being reported as performance
measures in PART.
The USPTO expects that in the future there will be surveys for the Academy participants
to take before they participate in the training program, immediately after they have
participated in the training program, and a follow-on survey that will be administered
between six months and one year following participation in the GIPA training programs.
The USPTO expects to use the pilot survey for the Academy alumni as the follow-on
survey after the pilot period is completed.
The USPTO will use the data gathered from the surveys to improve the programs
offered by the Academy and to expand its offerings.
There are no statutes or regulations requiring the USPTO to conduct this usage and
satisfaction measurement. The USPTO will use this questionnaire to implement
Executive Order 12862 of September 11, 1993, Setting Customer Service Standards,
published in the Federal Register on September 14, 1993 (Vol. 58, No. 176).
B. Survey Design
The pilot Global Intellectual Property Academy Alumni Survey is a one-time, voluntary
survey. This survey will primarily be conducted electronically via the Internet. However,
in cases of poor Internet connectivity or access restrictions, Word versions of the survey
will be e-mailed to the participants. The USPTO expects that the full survey will also be
conducted electronically via the Internet.
E-mail invitations containing a survey link will be sent to the pilot audience. The survey
link will then take the participants to the web-based Global Intellectual Property
Academy Alumni Survey. For those who wish to opt out of the survey, they simply need
to click on the built-in link provided in the e-mail invitation.
The survey link in the e-mail invitation is an individualized link that is tied to a particular
GIPA participant. This link is uniquely tied to the individual; therefore, the e-mail
invitation cannot be forwarded to another individual or another computer. The
participant will not need any passwords, user ids, or usernames to access the survey.
Once the individual is taking the survey, he or she can take the survey in more than one
sitting; their answers up to that point will be saved until they come back to the survey.
However, once the survey is actually submitted, that individual will not be able to take
the survey again. Once the survey is submitted, the data will reside on the third-party’s
servers.
All survey results will be kept confidential. Survey responses can be linked to the
participants and to the demographic data collected from them during the various
Academy training programs, but the actual data recorded from the survey will not be
directly linked to the participant. The data will reside on the third-party’s web server, but
the actual distribution of the data will be controlled by FCG. The USPTO will have
limited access to the data. The only data that can be accessed by the USPTO will be

2

the aggregated survey data and the frequency of the responses. FCG will have access
to individual survey responses for analysis purposes only.
The web-based Global Intellectual Property Academy Alumni Survey is hosted through
Survey Monkey, a third-party provider. The survey data will reside on Survey Monkey’s
protected servers. The FCG controls the distribution of the data through their account
with Survey Monkey. Since the data resides on a protected server and the distribution
of the data is controlled, there are no usernames or passwords needed to access the
survey data.
The Global Intellectual Property Academy Alumni survey will be available to all of the
foreign government officials who attended the United States-based Academy training
programs in 2008. During their participation in these programs, the USPTO collected email addresses. Since the GIPA survey will be distributed via e-mail, FCG will be able
to confirm e-mail addresses. The survey itself contains 37 questions.
C. Respondent Pool
The respondent pool for this survey is made up of all of the English-speaking foreign
officials who participated in the Academy training program based in the United States
during the 2008 calendar year. This universe was selected for the pilot in order to
gather the data needed to expand the survey to other groups of Academy participants
and to provide FY2008 performance measures to be reported to OMB for PART. Out of
702 foreign officials trained at the Academy during 2008, the USPTO estimates that 594
of them spoke English. The English-speaking participants represent 85% of all 2008
Academy participants. For the pilot survey, the respondent pool is limited only to those
foreign officials who participated in the 2008 Academy training program based in the
United States who speak English in addition to their own languages. Typically, the
largest percentage of total persons trained in the Academy programs speak English.
D. Sampling Method
At this time, the sampling method for the pilot survey is limited to all of the Englishspeaking foreign officials who participated in the 2008 Academy training programs;
therefore, everyone in the respondent pool will have the opportunity to participate in the
survey.
After the results from the pilot survey have been analyzed and a baseline established,
the USPTO plans to expand the survey to other groups who participate in the Academy
training programs. As a result, the sampling for the full survey is also expected to
expand. In addition, after the USPTO has had a chance to analyze the results of the
pilot survey, the agency plans to expand the survey to non-English speakers by
investing resources to develop foreign language versions of the GIPA surveys.

3

E. Collection Procedures
The Global Intellectual Property Academy survey is primarily available online and can
be accessed through the survey link that will be directly e-mailed to all of the Englishspeaking foreign officials who participated in the Academy training programs based in
the United States in 2008. Word versions of the survey will be available in special
cases. The e-mail invitation will provide an introduction to the survey, explain the
purpose of the survey, explain how the data collected from the survey will be used,
assure participants that their responses will be kept confidential, and inform them of the
deadline to complete the survey.
All of the potential respondents will receive the same survey. Several of the survey
questions are structured to have a random response order in order to diminish response
order bias. All of the potential respondents have been identified as having participated
in an Academy training program during the 2008 calendar year.
The survey will be open for a period of four weeks to allow FCG to gather the
responses. This should provide sufficient time to send reminders to those who haven’t
responded to the survey in order to maximize the response rate and generate a
representative response. This should also provide sufficient time for the FCG and
USPTO staffs to identify new e-mail addresses for those Academy participants who may
have changed their e-mail addresses within the last year.
F. Expected Response Rate
The USPTO has not surveyed this group before and therefore has no previous GIPA
survey study to use as a baseline for estimating the response rate for this pilot survey
and future surveys.
However, the USPTO expects that response rates will slightly vary by group. The
agency has done several surveys for many reasons in the last several years, such as a
2007 survey of Patent Applicants (0651-0052). For that study, the response rate was
as follows:
•

The median response rate from the four different response groups was 14%
o Large domestic corporations at 20%
o Small to medium-size businesses at 75%
o Universities and non-profits at 20%
o Independent inventors at 20%

The USPTO expects to have a higher response rate with the Academy participants.
The 0651-0052 rates were based on questionnaires that took longer than the Alumni

4

Survey to complete, and the agency expects that this will translate into a higher
response rate.
Another factor that should increase the response rate is that the Academy participants
are actively seeking information on intellectual property and their participation in the
Academy training programs is historically a sought-after attendance. From these facts,
the USPTO believes that participation in the GIPA Alumni will potentially be greater than
other USPTO surveys.
Additionally, the FCG has worked with other Government agencies, including the
Departments of State and Defense, to conduct surveys of international audiences. In
similar surveys, the FCG has seen response rates averaging around 36% for wellestablished surveys with a similar methodology (i.e., international audiences who have
participated in United States Government-sponsored education and training programs).
Based on this, the USPTO anticipates that 28% of the foreign officials who receive the
2008 GIPA Alumni survey will complete it, for 167 completed surveys out of a total 594
surveys. This rate is consistent with similar types of follow-on surveys conducted by
other training providers in the private sectors.
The USPTO estimates that it takes approximately 15 minutes (0.25 hours) to complete
the survey electronically. Based on the estimated number of responses and the
completion time for the survey, it is estimated that a total of 42 burden hours will be
associated with this pilot survey.
The USPTO believes that the audience for the GIPA training programs consists of highranking government officials, judges, lawyers, examiners, and others. The USPTO
believes that roughly 20% of the attendees fall into the high-ranking categories, while
the rest make up 80% of the attendees. The USPTO believes that the hourly labor rate
for 20% of the attendees would be roughly the equivalent of the professional hourly rate
of $310, while the rest would be roughly the equivalent of the para-professional rate of
$100. The professional hourly rate of $310 used to calculate the respondent costs for
this survey is the median rate for attorneys in private firms as published in the 2007
report of the Committee on Economics of Legal Practice of the American Intellectual
Property Law Association (AIPLA). This report summarized the results of a survey with
data on hourly billing rates. The para-professional hourly rate of $100 used for this
survey comes from the 2008 National Utilization and Compensation Survey Report,
published in March 2008 by the National Association of Legal Assistants (NALA).
Using these U.S. hourly rates, the USPTO estimates a professional hourly rate of $62
(20% of $310) and a para-professional rate of $80 (80% of $100). Adding these two
rates together provides the total hourly rate of $142. This is a fully loaded hourly rate.
Based on the estimated number of responses and the completion time for the survey,
the USPTO estimates that a total of 42 burden hours will be associated with this pilot
survey (0.25 hours x 167 responses = 42 hours). Multiplying that against the hourly rate

5

of $142 yields a total annual respondent cost burden of $5,964 (42 hours x $142 =
$5,964).
Please note that since individuals with varying job titles and pay grades from 127
countries attended GIPA training programs in 2008, the USPTO is unable at this time to
derive a concise international labor rate for these individuals. Due to this and the fact
that the training was conducted in the United States, the USPTO is using the
corresponding United States pay rates to calculate the hourly labor rates. If the agency
can obtain a more concise hourly labor rate data for these individuals, these rates will
be used to calculate the respondent burden in the full survey.
G. Follow-up Procedures Employed
For the pilot survey, survey reminders will be sent to those respondents who have not
completed and submitted their surveys. If needed, FCG will also conduct telephone
interviews with the non-respondents in order to generate a representative response
across the group. A script for these telephone interviews has been developed and is
included with this submission.
As an incentive to respond to the survey, respondents will be notified that their
responses will assist the USPTO in delivering effective intellectual property programs.
In addition to the assurances that their responses will remain completely confidential,
the USPTO will also assure the Academy alumni that their participation in this survey
will not effect their future participation in any Academy programs or other programs
sponsored by the U.S. Government.
The USPTO expects that these follow-up procedures will carry forward to the full
survey. If other possible follow-up procedures are identified while conducting the pilot
survey, they will be considered for the full survey submission.
H. Assurance of Confidentiality
The data collected from this survey will be confidential. The actual data will reside on
Survey Monkey’s protected server. The distribution of the data will be controlled by
FCG through their account with Survey Monkey. FCG will only report the aggregated
data and the frequency of the responses, not the individual responses or comments
about the survey. Agency personnel will not be able to view the individual responses or
data related to the survey. FCG will control the amount of access that the USPTO has
to the aggregated data.
The Global Intellectual Property Academy survey is an electronic survey that will be
completed online via the Internet. The Academy participants will need the survey link in
order to enter the survey, which will be directly e-mailed to them in the survey invitation.
For this survey, no passwords, user ids, or usernames will be needed to access the
survey. However, the survey URLs that are e-mailed to the participants will be
individualized links that are uniquely tied to them. This means that the survey cannot be
forwarded to another individual or to another computer. This unique link will be the only

6

way that the survey results can be linked back to the Academy attendee and the
demographic data which is being used to track the survey responses in order to ensure
that the survey findings are representative across all of the variables. The data that is
used to distribute the survey, survey reminders, and survey pre-notifications will be
maintained in a separate file from the quantitative data.
The survey will be in HTML format and will be accessed through Survey Monkey’s
website. Once a respondent submits their completed survey, the data will then reside
on Survey Monkey’s protected servers. FCG will then be able to control the access to
and distribution of the data.
The survey responses, the contact and respondent information, and the e-mail invitation
will reside in three different files in a secured folder on Survey Monkey’s server. One
file will contain the survey results, which will be created from the survey responses. The
second file will contain the contact information used to identify where to send the e-mail
invitations and the demographic data obtained from the GIPA participants. The third file
will identify the Academy participants who were sent the e-mail invitation and will
contain the e-mail invitation. These three files are relational files. While FCG will have
access to the individual responses, they will not report or share the responses. Only
aggregated data will be reported.
I. Analysis Plan
Analysis of data collected from the survey will be based upon response frequencies and
simple cross-tabulations. Use of response data for Academy performance measures is
based upon multi-variable components that are reported as indices. This approach is
currently used by other Departments, including the Department of State. Once the full
evaluation system is in place, it is anticipated that more rigorous statistical analyses will
be implemented.
Since this justification statement covers a pilot survey that the USPTO fully expects to
develop into a full-blown ongoing survey, the agency is including additional
methodology information in this generic clearance submission that is not generally
included. This additional information concerning compensating for non-response bias,
methods to maximize responses, the testing of procedures, and contact for statistical
aspects and data collection is provided below.
Compensating for Non-Response Bias
The USPTO has demographic data on the universe and has provided that data to FCG.
The demographic data includes the country of residence, program course, participation
dates, and the sex of the participant. The FCG will be tracking these variables while the
survey is open to ensure that the final response total is representative across them.
The survey invitations contain a link that is uniquely tied to that particular respondent.
The survey is also linked to the demographic data. This allows FCG to track the survey
responses against the variables to ensure that a representative total is being generated.

7

The unique survey link also allows FCG to see who has responded to the survey in realtime. For the pilot survey, the FCG expects to track the country of residence and the
program course variables.
If it becomes apparent that the responses are not representative under one or more of
the variables, the FCG staff will send survey reminders to those individuals who have
not yet completed their surveys to prompt additional responses. If sending the survey
reminders fails to generate a representative response, FCG staff will contact the nonrespondents by phone in an attempt to obtain their responses. The FCG may also work
with GIPA staff to interview or conduct a focus group with the non-responding group to
determine if their responses would differ from those who have responded to the survey,
although it is unlikely that this course will need to be taken. Since the survey will be
based upon a census approach for the English-speaking universe, it is expected that a
representative response will result.
It is anticipated that once the survey system moves into full operation after this pilot,
individualized sample designs for each course will be developed.
The pilot survey also contains questions for other demographic data, including
Intellectual Property focus, years in IP work, and organization that the USPTO does not
currently track. The USPTO will be using these questions to establish baseline data on
the respondent universe.
Methods to Maximize Responses
In order to maximize the number of responses received from the survey, the USPTO
intends to do the following:
•

Provide a pre-survey notification to foreign officials through the Embassies.
Since the pilot survey is aimed at the foreign officials who have attended the
Academy training programs based in the United States, the USPTO believes
that a pre-notification about the survey coming from a recognizable contact at
the particular Embassy will make the recipient more receptive to the survey than
just receiving the e-mail invitation without a prior introduction. The USPTO
expects to continue using this approach once the survey is in full production.

•

Assure all participants that the responses will be kept confidential and only
aggregated data will be used.

•

Conduct the survey using the FCG, an independent evaluation organization.

•

Conduct the survey via e-mail, a known and convenient method of
communication with the participant group. FCG staff will be tracking the e-mail
invitations that are sent out. If the e-mail addresses are bad, the e-mail in
question will be “bounced” back to FCG, who may track these particular e-mails
for the percentage of bad e-mail addresses. If there is a significant number of

8

bad e-mails, the USPTO will contact the Embassies to see if new e-mail
addresses for those individuals can be determined.
•

Provide follow-up reminders periodically to those individuals who have not
responded to the survey.

The survey questions are primarily close-ended and brief to allow participants to answer
the questions quickly and easily, even if English is their second language. All
respondents will also receive the name and number for a GIPA contact to answer any
questions they might have about the survey. Once the survey moves into full
production, the data collection instruments will be pre-tested with test respondents
similar to the evaluation target audience.
Testing of Procedures
This pilot survey will be used by the USPTO and FCG to develop and test the survey
methodology for the full-blown, ongoing surveys for the Academy training programs.
FCG staff will use the pilot survey to test the questions for clarity, brevity, relevance,
user-friendliness, understandability, and sensitivity. The pilot survey will also be used to
see whether most respondents are willing to answer the survey questions. Based on
the survey feedback, FCG will clarify instructions and questions, refine the response
categories, and add new questions as necessary. The FCG’s team of professional
evaluators have been conducting survey design, sampling, and data analysis for federal
clients an average of 15 years each.
Once the full survey methodology has been determined and the ongoing survey
established, FCG staff will continue pre-testing the survey to clarify the instructions and
questions, refine the response categories, and add new questions as necessary.
Individual follow-up telephone conversations with selected respondents may be
conducted. Focus groups, with individuals meeting to go over the survey instruments,
may also be used for pre-testing. It is anticipated that the focus groups will be
conducted with current classes offered at the Academy so that additional costs will not
be incurred.
The research topics in the survey will be organized according to the widely accepted
Kirkpatrick model for evaluating training impact (reaction, learning, behavior change,
and organizational results). This evaluation approach, developed in 1959 by Donald
Kirkpatrick, the former President of the American Society for Training Development, is a
four-level approach which measures participant reaction, learning, and application of
behavior and results and is well-established in the United States and overseas. The
specific questions are based upon models that have been developed by evaluation and
survey design specialists, social science research analysts, and organizations, including
Knowledge Advisors, Toyota Motor Sales, Defense Acquisition University, Caterpillar,
the U.S. Department of State, the U.S. Department of Defense, Cisco Systems, and
Grupo Iberdrola (Spain). In particular, the question models have been successfully
tested and deployed to over 10,000 U.S. sponsored international visitors and foreign

9

officials since 2003. While the question construction and response scales have been
utilized with these audiences, the response categories have been adapted to address
specific topic areas for Intellectual Property protection and enforcement. Questions also
focus on each of the particular areas for which the Academy provides training, including
Patents, Trademarks, Copyright, and Enforcement.
Contact for Statistical Aspects and Data Collection
The Office of Intellectual Property Protection and Enforcement (OIPPE) of the USPTO
is the sponsor of this survey. They are using an external, independent evaluation
organization, the Federal Consulting Group (FCG), to conduct the survey. FCG has
team members who specialize in statistics to assist with research design, data
collection, and analysis. FCG is also the executive agent for the American Customer
Satisfaction Index (ACSI) for the federal government. The point of contact at the
USPTO is Peggy Orser.
She can be reached at (571) 272-9300 or
Peggy.Orser@uspto.gov. The contact at FCG is Ted Kniker, who is a member of
OMB’s impact evaluation panel, under the Performance Improvement Council. He can
be reached at (202) 513-7656 or Ted_Kniker@nbc.gov.

10