SECTION B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
Universe and Respondent Selection
The Commercial Service (CS) wants to capture user satisfaction data for advocacy services provided. User populations will have already used the specific product/service.
For most survey instruments currently in use, all clients are surveyed. Due to the small number of clients participating in advocacy services, all users of advocacy services will receive a user satisfaction survey. However, if response rates are insufficient, the CS has designed sampling methodologies. Where product/service user populations are large and response rates are sufficiently high, CS favors simple random samples without replacement.
The estimated potential respondent universe is 650 companies and we expect that based on past response rates, the advocacy user satisfaction survey will achieve a 60% response rate or approximately 390 survey responses. Obtaining information regarding advocacy services provided to companies, due to the fact that each target market or country is different and therefore it is important to understand the overall customer satisfaction with services provided in each country that the Commercial Service operates. There is also limited data for each country so surveying the whole population will enable the Commercial Service to obtain as much feedback as possible.
Procedures for Collecting Information
Due to the small number of clients participating in advocacy services, all users of advocacy services will receive a user satisfaction survey.
As the purpose of Commercial Service user satisfaction surveys is to evaluate client perceptions of quality and value with respect to the advocacy services provided to US companies and then to interpret the data to make business process improvements, market research states that user satisfactions surveys that receive a 50% response rate are sufficient to implement policy and business process improvements to address customer satisfaction.
Methods to Maximize Response
The data collection method chosen for the advocacy quality assurance survey is an e-mail message delivering a hot link to a web enabled survey – in part, because this allows for minimum expense and should yield more considered answers. If the client does not respond to the survey within two weeks, another e-mail reminder is sent to the client. A number of efforts have already been undertaken to improve response rates. An Internet delivered survey reduces the burden on the respondents, surveys have been shortened, and open-ended questions requiring narrative responses have been reduced. As a way to increase response rates, our domestic and overseas staff informs their clients that once a service is completed, they will receive a user satisfaction survey via e-mail within two weeks. In addition, returned e-mails are addressed systematically in order to increase response rates. Ongoing review of the survey instruments and policies may result in further refinements and resubmission of instruments. If changes are deemed necessary to improve response rates or more accurately capture client perceptions, these changes may occur within the five-year approval.
To address non-response rate, when the response rate is lower than recommended, the CS will perform telephone interviews of non-respondents to inquire about their user satisfaction and complete the user satisfaction survey by phone.
OMB’s TERMS OF CLEARANCE provided on 9/21/2005: “This survey is approved for 18 months. Prior to resubmission, the agency should develop an analysis of nonresponse bias for the survey and for each important question on the survey. If significant bias is identified, the agency must propose changes to the survey to accommodate this bias and/or correct the results in a systematic manner.” Since the CS has not yet conducted the Advocacy Quality Assurance Survey, we are unable to provide an analysis of nonresponse bias for this survey. The CS has not yet conducted the Advocacy Quality Assurance Survey due to a lack of IT resources that are needed to build a database for tracking Advocacy cases that do not involve government procurements. The CS is working to free up IT resources for this database so that we can begin surveying these clients in near future. Once we begin surveying Advocacy clients, we will provide OMB with an analysis of nonresponse bias.
Testing Procedures
Questionnaire construction is one of the most critical stages in the survey development process so a great deal of thought has gone into the design issues involved (e.g. development and pretesting) regarding the “Advocacy Quality Assurance Survey.” In order to prevent the resulting data from being seriously misleading, the CRM Unit has worked with advocacy specialists within the organization to make sure that the concepts be clearly defined and survey questions unambiguously phrased. Furthermore, designing a suitable questionnaire requires attention be given to its length and the order in which questions are asked – long questionnaires are apt to induce respondent fatigue and errors arising from inattention, incomplete answers, and refusals. In addition to making sure that the questionnaire is the suitable length, all questions in the survey have been pretested to ensure that they are understood by the respondents and do not adversely affect survey cooperation. The survey was based on a modified version of a proven survey instrument. The core of the survey will entail questions of customer satisfaction and perception of quality of advocacy services provided by the CS. CS also tests on external clients whenever appropriate.
Contacts for Statistical Aspects of Data Collection
Individuals who will be or have been consulting on the statistical aspects of the design of the advocacy user satisfaction survey, as well as those individuals who will actually collect and/or analyze the information, are:
- Suzan Winters
Customer Relationship Management Unit
U.S. Commercial Service
U.S. Department of Commerce
(202) 482-6042 fax: (202) 482-2599
- Joseph P. Carter
Customer Relationship Management Unit
U.S. Commercial Service
U.S. Department of Commerce
(202) 482-3342 fax: (202) 482-2599
LEGAL AUTHORITY: 15 U.S. C 4724
| File Type | application/msword |
| File Title | SUPPORTING STATEMENT – SECTION B |
| Author | joseph carter |
| Last Modified By | joseph carter |
| File Modified | 2007-04-06 |
| File Created | 2007-04-06 |