Supporting Statement B
“USPTO Website Customer Satisfaction Surveys”
United States Patent and Trademark Office
Office of the Chief Information Officer
OMB Control No. 0651-New
STATISTICAL METHODS
Universe and Respondent Selection
All surveys covered under this clearance will have these specific characteristics:
They will be conducted using the ForeSee CXA methodology.
They will measure customer satisfaction with United States Patent and Trademark (USPTO) websites and related media.
Only a small percentage of each website’s visitors will take the survey.
The survey will be served up randomly as visitors experience the website.
Collection of personal information through the surveys is not required by the ForeSee CXA methodology.
The USPTO will be the sole owner of the data results.
The ForeSee CXA model measures multi-variable components that are reported as indices. Various models have been developed to meet the objectives of each type of website. The models contain between 3-25 questions depending on the needs of the Agency for additional custom questions. The actual number of questions presented can be decreased in order to maximize response rates and minimizes obtrusiveness to website visitors. To accomplish this, imputation statistical methodology is used to estimate the missing values in a data set based upon the information that is available. Because multiple questions are used for each element of the econometric model, stable results are obtained using a sample size of 300 respondents.
While there is a quota of 300 respondents per sample, the selection of each respondent is made as a random probability selection from the universe of visitors on the website at any given time. Thus, the maintenance of random sampling is at the individual level rather than at the total sample level. The historical response rate to these surveys of government websites conducted under this clearance ranges between 1.5% and 35% and has averaged about 5% in the last 3 years. These results are comparable to the response rates for surveys of private sector websites. No follow-up is attempted if a respondent fails to complete a portion of or the entire questionnaire.
Procedures for Collecting Information
Data will be collected through online surveys that will be presented randomly to website visitors. Survey respondents are identified through a number of conditions that are contingent on the website’s traffic and architecture. The trigger code – which causes the survey to be presented – has a number of options that can be customized to the specific website. There is a variable that causes the survey to be presented randomly to a percentage of website visitors as they experience the website. The trigger can also utilize another variable that is referred to as a loyalty factor. This would prevent a visitor from receiving the survey before seeing more than a specified number of pages. The surveys will run continuously over the time period specified by the Agency. The USPTO will be able to access and analyze all data that is collected over the time period.
The surveys are presented continuously throughout the subscription period, due to the continuous updating of the websites. The USPTO must monitor public perception of their websites through the various scores and analysis that they receive.
No personal or demographic information of the respondents is acquired through the ACSI methodology on the surveys. The Agency has the option to add custom questions that are deemed necessary or beneficial in understanding respondent concerns and priorities.
An online reporting facility will be available for government personnel to access the results of the data collection. The facility is hosted on a secure remote server, and a username and password are established for clients to retrieve their data. All data will be owned by the USPTO. The results are used to create indices, which are compiled from aggregated data and measurements.
The total respondent sample is 13 X 7,700 ≈ 100,000. This number is derived from the estimated number of surveys being conducted (13) multiplied by the estimated number of respondents per year (7,700).
Methods to Maximize Response
To maximize the response rate, most of the surveys are short and take only 2-3 minutes to complete, but a minority of surveys may take up to 10-15 minutes to complete. Questions are brief and easy to answer. The welcome text will indicate that the data is being collected by an independent, third party; that the purpose of the specific survey is to improve the citizen satisfaction with the Agency website; and that no data will be used for sales calls or other purposes. Also, surveys are presented only to a small sample of the visitor universe yet provides truly actionable information. Citizens offered a survey are not likely to receive more than one invitation to complete it during the subscription period of one year.
Because the index approach employs multiple questions to create the index and because the 1- 10 rating scale used for the majority of the questions generates a mean (as compared to a proportion) which is then converted to a 0 – 100 scale, and because we know empirically that the standard deviation tends to be approximately 20 for this survey data, a sample size of approximately 300 yields confidence intervals in the range of +/- 1.5 to +/- 3.5 at the 95% confidence level and confidence intervals of +/- 2.3 to +/- 2.6 at the 90% confidence level on the 0 – 100 scale.
The intended purpose of these data collections is to guide leaders and managers in making managerial decisions about ways to improve the quality of government websites and experience by visitors to the sites. Data collected in these surveys will not be used to make policy decisions.
Testing of Procedures
Those involved with the ForeSee CXA methodology are international leaders in assessing and understanding customer satisfaction. The current ForeSee Chief Research Scientist, Dr. Jae Cha, worked closely with Dr. Claes Fornell, Donald C. Cook Professor of Business Administration, Emeritus, at the University of Michigan and creator of the American Customer Satisfaction Index (ACSI) of which the ForeSee CXA methodology is a derivative. The ultimate purpose of the ACSI is to help improve the quality of goods and services available to American citizens, while the ultimate goal of using the ForeSee CXA methodology is to improve the quality of government websites to American citizens.
The ForeSee CXA methodology produces an econometric model that enables agencies to obtain insights for valuable, high-return, customer-focused decisions. An important advantage, in contrast to methods that rely solely on survey questions, is that it produces responses with statistical stability and low-chance variation. This helps ensure uniform and consistent results.
No tests of procedures or methods will be undertaken. The ForeSee CXA methodology is patented, and the standard questionnaire has undergone extensive, rigorous testing and study in academia and through empirical studies with several million respondents in both government and the private sector. More specifically, the E-Government website surveys have been used in the Federal Government at numerous agencies over the past 14 years and also have undergone extensive testing to identify the set of questions that increase reliability and utility while reducing burden.
Contacts for Statistical Aspects and Data Collection
Questions regarding any statistical aspects employed or data collection procedures used should be directed to:
Jae Cha, Chief Research Scientist
ForeSee
2500 Green Road, Suite 400
Ann Arbor, MI 48105
Telephone: (734) 352-6228
Email: jae.cha@answers.com
Web: www.ForeSee.com
Administrative questions regarding the use of this generic clearance by the USPTO should be directed to:
Mike Jennings
United States Patent and Trademark Office
600 Dulany St,
Alexandria, VA 22314
Telephone: (571)-272-9739
Email: Michael.Jennings1@USPTO.GOV
| File Type | application/msword |
| File Title | Section B |
| File Modified | 2018-10-29 |
| File Created | 2018-10-29 |