Download:
docx |
pdf
SUPPORTING STATEMENT
U.S.
Department of Commerce
International
Trade Administration
Surveys
for User Satisfaction, Impact, and Needs
OMB
Control No. 0625-0275
Collections
of Information Employing Statistical
Methods
Describe (including a numerical estimate) the potential respondent
universe and any sampling or other respondent selection method to
be used. Data on the number of entities (e.g., establishments,
State and local government units, households, or persons) in the
universe covered by the collection and in the corresponding
sample are to be provided in tabular form for the universe as a
whole and for each of the strata in the proposed sample. Indicate
expected response rates for the collection as a whole. If the
collection had been conducted previously, include the actual
response rate achieved during the last
collection.
All
clients who have received assistance from ITA will have the
opportunity to fill out a transactional survey (i.e. comment card).
The transactional comment card survey is not mandatory or predicated
on receiving ITA assistance. The estimated number of clients
that will have the opportunity to fill out a transaction comment card
survey will be 30,000. Moreover, the estimated potential
respondent universe will be around 6,300 based on a historic average
response rate of 21 percent. Please see the table below for
respondent universe and historic response rates.
Fiscal
Year
|
Surveys
Sent
|
Responses
|
Percentage
|
2020
|
57,541
|
5,519
|
10%
|
2019
|
15,349
|
2,399
|
16%
|
2018
|
17,026
|
2,919
|
17%
|
2017
|
12,277
|
2,760
|
22%
|
2016
|
10,826
|
2,218
|
20%
|
2015
|
10,655
|
2,512
|
24%
|
2014
|
6,030
|
1,462
|
24%
|
2013
|
5,062
|
1,528
|
30%
|
2012
|
6,904
|
1,427
|
21%
|
2011
|
9,107
|
2,108
|
23%
|
2010
|
20,187
|
4,758
|
24%
|
2009
|
19,190
|
4,486
|
23%
|
2008
|
10,878
|
2,551
|
23%
|
In
addition, clients will be randomly selected to receive an annual
survey. The sampling methodology will be explained in further
detail in question 2 below. The annual survey is also voluntary and
sent to an average of 17,583 clients. The average response rate
is 14 percent, based on previous years’ annual survey results.
Please see the table below for respondent universe and historic
response rates.
Fiscal
Year
|
Surveys
Sent
|
Responses
|
Percentage
|
2020
|
26,619
|
2,459
|
9%
|
2019
|
27,787
|
2,279
|
8%
|
2018
|
16,110
|
1,119
|
7%
|
2017
|
12,338
|
1,646
|
13%
|
2016
|
15,029
|
4,057
|
27%
|
2015
|
7,616
|
1,263
|
17%%
|
Describe the
procedures for the collection of information
including:
Only
those selected to receive the annual survey will consist of a sample
selection. Clients that have received ITA assistance during the
fiscal year will be selected to receive the annual survey. The
sampling frame will use a random sample at the overall organizational
level and then use a stratified, random sample plan that takes into
account the relevant components of its clients such as industry, size
of business (perhaps number of employees or revenues), and/or
geographic location. To provide a simple example, please see Table 1
(page 9). Assume that the relevant components are industry and size.
Further, assume that there are four industries and two sizes of
business per industry. ITA will need to know what the proportions of
these four industries and two business sizes are across its sampling
frame. For purposed of this example, assume that Industry 1
represents 50% of the client base and Industries 2, 3, and 4
represent 25%, 15% and 10%, respectively.
Perhaps
another relevant indicator of interest may be the size of the
business or organization. Size could be an indication of the number
of employees a client has, or it could be total revenues, assuming
clients are willing to report the number or it is publicly available.
ITA will need to know the distribution of business sizes by industry
– or if it chooses other variables to segment its clients. For
purposes of this simplified example, assume the distribution is as
shown in Table 1.
The
extent to which ITA will be in a position to enhance the accuracy of
the estimates depends on the selection of the segments it defines.
Serious consideration needs to be given to a geographic component in
the segmentation scheme since ITA manages its accounts and cases from
geographically dispersed locations. Externalities unique to
geographic areas may have an effect on accounts/cases and should be
taken into consideration. In addition, having data elements such as
the congressional district identified and associated with each
account will facilitate decisions about how to respond to requests
for results at the congressional district level.
The
sample pool will be randomly selected and stratified using the
following parameters: industry, size of business (number of employees
or revenues), and geographic location. Each parameter will have equal
weight and be proportional to the overall sample population.
Additionally, the sampling procedure will adjust for over sampling to
ensure that the appropriate segments (industry, business size and
geographic location) are statistically represented in the respondent
sample size. The annual survey will conclude once the sample size is
statistically valid at the 95 percent confidence level.
All
other surveys will not be using any sampling procedures and ITA hopes
to achieve a 14 percent response rate, which is in line with its
historic averages. ITA will be taking the following steps to improve
its response rates: reducing the length of the Comment Card and
ensuring that all other surveys are as short as possible, revising
the content of the messages that will contain the survey links,
changing the subject line and sending a pre-survey message prior to
the survey being distributed.
Describe
methods to maximize response rates and to deal with issues of
non-response. The accuracy and reliability of information collected
must be shown to be adequate for intended uses. For collections
based on sampling, a special justification must be provided for any
collection that will not yield "reliable" data that can
be generalized to the universe
studied.
ITA
will be committed to employing the following best practices to
increase response rates:
Letting
clients know that ITA will be soliciting their feedback at the
beginning of a “case” because ITA values their
perspective to inform its continuous improvement efforts.
Engaging
the client in determining an appropriate timeframe for capturing
their feedback after a meaningful service has been delivered.
Facilitating
the client providing the feedback via a method of their choice –
for example, via telephone interview, or email invitation to take
a web survey.
Providing
pre-notification that a customer feedback survey is coming –
this can be via a phone call (if the client has indicated a
preference for doing the telephone interview) and/or via an
advance email letting the client know when to expect the
invitation and what the SUBJECT of the email will be and what
FROM address will be indicated. (In the private sector, this
pre-notification has been shown to be effective when it comes
from a known or trusted source.)
Ensuring
that the ‘invitation’ language explains why ITA is
conducting the survey, why it is important the selected client
participate in the survey, how it intends to use the data
collected and when the aggregated results will be shared.
Sending
reminders to encourage the client to participate.
Sharing
the aggregated results with all clients – for example, the
results can be shared periodically on a website or via an email.
Communicating
changes/improvements that are being made in response to customer
feedback surveys.
For
the annual survey, the sampling procedure will adjust for over
sampling in order to yield "reliable" data that can be
generalized to the universe studied. That is, ITA will continue to
send additional surveys from its sample pool until the appropriate
response rate is attained to be statistically valid at the 95 percent
confidence level for the universe studied.
ITA
will conduct an analysis of nonresponse bias for the survey and for
each important question on the survey. If significant bias is
identified, ITA will propose changes to the survey to accommodate
this bias and/or correct the results in a systematic manner. However,
ITA has not observed any bias in its survey responses and has not
conducted a nonresponse bias analysis for this survey. ITA will
conduct this analysis as soon as it’s warranted.
Describe any tests of procedures or methods to be undertaken.
Testing is encouraged as an effective means of refining collections
of information to minimize burden and improve utility. Tests must
be approved if they call for answers to identical questions from 10
or more respondents. A proposed test or set of tests may be
submitted for approval separately or in combination with the main
collection of information.
A
cross-tabulation and regression analysis will be employed for the
annual survey results only. The purpose of this test is to determine
drivers of customer satisfaction and will be for internal use only.
More specifically, ITA will quantitatively test the questionnaire
itself in terms of:
gaining insight into whether
there are apparent relationships between/among satisfaction/use
again/recommend questions and the needs and impact questions
the effectiveness of trying to
capture all three types of data (satisfaction/use again/
recommend, needs and impact) in one survey questionnaire
utility of individual
questions,
understanding if there are
topics that customers think should be included that were not in
the pilot questionnaire
obtaining feedback from
respondents about the questionnaire experience itself
gaining understanding about the
willingness/ability of respondents to report on impact results
Provide the name and telephone number of individuals consulted on
statistical aspects of the design and the name of the agency unit,
contractor(s), grantee(s), or other person(s) who will actually
collect and/or analyze the information for the
agency.
Individuals
who will be or have been consulting on the statistical aspects of the
design of the advocacy user satisfaction survey, as well as those
individuals who will actually collect and/or analyze the information,
are:
Joe
Carter, International Trade Administration, U.S. Department of
Commerce: (303) 844-5656
John
Seo, International Trade Administration, U.S. Department of Commerce:
(202) 809-5977
LEGAL
AUTHORITY: PUBLIC LAW 15 U.S.C. et seq and 15 U.S.C. 171 et seq
Table
1
(NOTE:
All data are for illustrative purposes only)
|
Industry
1
|
Industry
2
|
Industry
3
|
Industry
4
|
|
Proportion
of All Clients (A)
|
50%
|
25%
|
15%
|
10%
|
|
Business
Size
|
%
Large
|
%
Small
|
%
Large
|
%
Small
|
%
Large
|
%
Small
|
%
Large
|
%
Small
|
|
Proportion
within the Industry (B)
|
90
|
10
|
80
|
20
|
13
|
87
|
8
|
92
|
|
Proportion
for Stratified, Random Sample (C)
|
45.00%
|
5.00%
|
20.00%
|
5.00%
|
1.95%
|
13.05%
|
0.80%
|
9.20%
|
|
|
|
|
|
|
|
|
|
|
|
Proportion
of Respondents Reporting Increased Revenues (D)
|
35.00%
|
30.00%
|
40.00%
|
52.00%
|
32.00%
|
18.00%
|
22.00%
|
17.00%
|
|
Total
Amt of Increased Revenues Reported (E)
|
$110,000,000
|
$20,000,000
|
$15,000,000
|
$8,000,000
|
$75,000,000
|
$25,000,000
|
$8,817,000
|
$27,000,451
|
$288,817,451
|
Total
Respondents Reporting Revenue Increases (F)
|
20
|
30
|
19
|
31
|
18
|
32
|
17
|
37
|
204
|
Average
Amt of Increased Revenues for this Segment (G)
|
$5,500,000
|
$666,667
|
$789,474
|
$258,065
|
$4,166,667
|
$781,250
|
$518,647
|
$729,742
|
|
|
|
|
|
|
|
|
|
|
|
Total
Number of Clients for this Segment or "Cell" (H)
|
4500
|
500
|
2000
|
500
|
195
|
1305
|
80
|
920
|
(S)
10,000
|
Total
Number to Apply Avg to (J)
|
1575
|
150
|
800
|
260
|
62
|
235
|
18
|
156
|
|
Total
Estimated Revenue Increased for Segment (K)
|
$8,662,500,000
|
$100,000,000
|
$631,578,947
|
$67,096,774
|
$260,000,000
|
$183,515,625
|
$9,128,188
|
$114,131,636
|
$10,027,951,171
|
Calculations
in support of Table 1
Total
number of clients in sampling frame x proportion industry is of total
sampling frame x proportion business size is of the industry = number
of clients in that segment (S x A x B = H)
Total
amount of Increased Revenue (or number of Jobs Created or number of
Jobs Safeguarded) reported / number or respondents reporting
Increased Revenue (or Jobs Created or Jobs Safeguarded) = average
Increased Revenue (or Jobs Created or Jobs Safeguarded) among those
reporting Increased Revenue (or Jobs Created reported or Jobs
Safeguarded reported)(E/F=G)
Total
number of clients for the segment x proportion of reporting Increased
Revenues (or Jobs Created or Jobs Safeguarded) = total number to
apply the average amount of Increased Revenue (or Jobs Created or
Jobs Safeguarded) to (H x D = J)
Average
amount of Increased Revenues for this segment (or Jobs Created or
Jobs Safeguarded) x total number to apply the average to = total
estimated Increased Revenue (or Jobs Created or Jobs Safeguarded) (G
x J = K)
Once
the total estimated Increased Revenues (or Jobs Created or Jobs
Safeguarded) are determined for each segment (or cell), sum those to
have Total Estimated Revenue Increased (or Jobs Created or Jobs
Safeguarded)
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | DOC PRA TOOLS 2020 |
Subject | 2020 |
Author | Dumas, Sheleen (Federal) |
File Modified | 0000-00-00 |
File Created | 2021-10-04 |