COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS.
Universe and Respondent Selection
All surveys covered under this clearance will have these specific characteristics:
They will be conducted using the ACSI methodology.
They will measure customer satisfaction with federal government websites.
Only a small percentage of each website’s visitors will generally be qualified to take the survey.
The criteria for qualification will vary by agency and will determine when and where the survey will be presented.
The survey will be served up randomly as visitors experience the website.
Collection of personal or demographic information through the surveys is not required by the ACSI methodology used by ForeSee Results.
Government agencies will be the sole owner of the data results. ForeSee Results has the right to create and publish aggregate indices.
The ACSI model measures multi-variable components that are reported as indices. Various models have been developed to meet the objectives of each type of website. The models contain between 25-40 questions depending on the needs of the agency for additional custom questions. ForeSee Results is able to decrease the actual number of questions presented which maximizes response rates and minimizes obtrusiveness to website visitors. To accomplish this, they use an imputation statistical methodology to estimate the missing values in a data set using the information that is available. Because multiple questions are used for each element of the econometric model, stable results are obtained using a sample size of 300 respondents.
While there is a quota of 300 respondents per sample, the selection of each respondent is made as a random probability selection from the universe of visitors on the website at any given time. Thus, the maintenance of random sampling is at the individual level rather than at the total sample level. The historical response rate to the ForeSee Results surveys of government websites conducted under this clearance ranges between 1.5% and 35% and has averaged about 5%. These results are comparable to the response rates for surveys of private sector websites. No follow-up is attempted if a citizen selected to take a survey fails to complete a portion of or the entire questionnaire.
Procedures for Collecting Information
Data will be collected through an on-line survey that will be presented to website visitors randomly. Virtually every survey will require a unique set of questions and an individualized sample design.
Survey respondents are identified through a number of conditions that are contingent on the website’s traffic and architecture. The trigger code – which causes the survey to be presented – has a number of options that can be customized to the specific website. There is a variable that causes the survey to be presented randomly to a percentage of website visitors as they experience the website. The trigger can also utilize another variable that is referred to as a loyalty factor. This would prevent a visitor from getting the survey before seeing more than a specified number of pages. The survey will run continuously over the time period specified by the agency. The agency will be able to access and analyze all data that is collected over the time period.
The survey is presented continuously throughout the subscription period, because the Internet is constantly changing and websites are continuously updating. Government agencies must have a steady pulse on what is taking place on their website through the various scores and analysis that they receive from ForeSee Results. Another reason for the survey to run continuously is that it is not possible to know how quickly the necessary data will be collected or how often agency management must have results.
No personal or demographic information of the respondents is acquired through the ACSI methodology on the survey. The government agency has the option to add custom questions that are deemed necessary or beneficial in understanding citizen concerns and priorities. Often those questions are demographic in nature.
An on-line reporting facility will be available for government personnel to access the results of the data collection. The facility is hosted on a secure remote server, and ForeSee Results establishes a username and password for clients to retrieve their data. All data will be owned by the government agency involved. ForeSee Results will use the results to create indices, which are compiled from aggregated data and measurements.
Projected estimates for fiscal years 2008 – 2010 are as follows:
The total sample is 200 X 5,000 = 1,000,000. This number is derived from the number of surveys being conducted (200) multiplied by the estimated number of respondents per survey per year (5,000).
The total sample is 225 X 5,000 = 1,125,000. This number is derived from the number of surveys being conducted (225) multiplied by the estimated number of respondents per survey per year (5,000).
The total sample is 250 X 5,000 = 1,250,000. This number is derived from the number of surveys being conducted (250) multiplied by the estimated number of respondents per survey per year (5,000).
Methods to Maximize Response Rates
To maximize the response rate, the surveys are kept short and take only 2-3 minutes to complete. Questions are brief and easy to answer. The welcome text will indicate that the data is being collected by an independent, third party; that the purpose of the survey is to improve the citizen satisfaction with the agency website; and that no data will be used for sales calls or other purposes. Also, the survey is presented only to a small sample of the visitor universe yet provides truly actionable information. Citizens offered a survey are not likely to receive more than one invitation to take the survey during the subscription period.
Testing of Procedures
No tests of procedures or methods will be undertaken. The ACSI methodology is patented and has undergone extensive academic rigor and study. It has already been used for fourteen years with great success as a key quality metric for the U.S. economy and in over 20 foreign countries. Most importantly, the ACSI has become a key quality performance metric for the Federal Government in numerous studies conducted over nine years.
Contact for Statistical Aspects and Data Collection
Questions regarding any statistical aspects employed or data collection procedures used should be directed to:
Russ Merz, PhD, Research Director
ForeSeeResults
625 Avis Drive, Suite 200
Ann Arbor, MI 48108
Telephone: (734) 205-2604
Fax: (734) 205-2601
Email: Russ.Merz@foreseeresults.com
Web: www.foreseeresults.com
American Customer Satisfaction Index (ACSI)
Professor Claes Fornell, Director
National Quality Research Center
Stephen M. Ross School of Business
University of Michigan
Ann Arbor, MI 48109-1234
Telephone: (734) 763-9767
Fax: (734) 763-9768
Email: cfornell@umich.edu
Web: www.theacsi.org
Administrative questions regarding the use of this generic clearance by the U.S. Treasury’s Federal Consulting Group should be directed to:
Ronald Oberbillig
Chief Operating Officer
Federal Consulting Group
799 9th St, NW
Washington, DC 20001
Telephone: (202) 504-3656
Fax: (202) 504-3628
Email: Ron.Oberbillig@bpd.treas.gov
Web: www.fcg.gov
| File Type | application/msword |
| Author | bjinnoha |
| Last Modified By | bjinnoha |
| File Modified | 2008-09-25 |
| File Created | 2008-09-25 |