NOAA Engagement tool Supporting_Statement_071810 Part Brev

SUPPORTING STATEMENT
NOAA Constituent Engagement Survey
OMB CONTROL NO. 0648-xxxx

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of
entities (e.g. establishments, State and local governmental units, households, or
persons) in the universe and the corresponding sample are to be provided in tabular
form. The tabulation must also include expected response rates for the collection as
a whole. If the collection has been conducted before, provide the actual response
rate achieved.
Potential respondents are members of known constituent groups located within the states
of Texas, Louisiana, Mississippi, Alabama or Florida. The survey will be implemented
by using an Indefinite Duration, Indefinite Quantity (IDIQ) contractor for NOAA who
specializes in survey administration and analysis.
*Constituent Group
Non-profit organizations
Other Federal Agencies
NOAA Keystone Partners
State and local governments
K-12 Education Professionals
Higher Education Professionals
Business and Industry
Total
Expected response rate

Estimated
Sample
100
200
300
400
400
400
800
2,600
25%

Estimated Number of
Completed Surveys
25
50
75
100
100
100
200
650

* The response rate and the public burden hours are estimates which are specific to the NOAA Gulf of Mexico
Region. Other NOAA regions may have a different sample population and have different estimated response rates
and public burden hours.

2. Describe the procedures for the collection, including: the statistical methodology for
stratification and sample selection; the estimation procedure; the degree of accuracy
needed for the purpose described in the justification; any unusual problems requiring
specialized sampling procedures; and any use of periodic (less frequent than annual) data
collection cycles to reduce burden.
The NOAA Engagement Program staff will collect the potential respondents’ contact
information by working with Gulf of Mexico Sea Grant Programs, NOAA’s Gulf of Mexico
Regional Collaboration Team, Local Chambers of Commerce and National Marine Educators
Association to compile a database containing the physical mailing address, phone number and
email address of respondents. The sample population is divided into seven subgroups (see #1
above) that are more homogeneous as individual groups than as a whole population; therefore we
1

will use a stratified sampling method. Potential respondents with email addresses will be
contacted by email, and provided a link to the survey at SurveyMonkey.com. Those without an
email address will receive the survey by mail, which can be returned by mail or facsimile. As
noted in the table (see #1 above), more samples will be taken in subgroups that are larger, in
subgroups with greater variability, and in subgroups where we expect to obtain higher response
rates.
Upon completion, all data will be collected into a database system where results will be
processed and monitored. All post-statistical analysis will be conducted by a non-biased third
party. This outside firm will employ a variety of analytical tools, including, but not limited to,
correlation and regression analysis, t-testing and cross-tabular analysis.
3. Describe the methods used to maximize response rates and to deal with non-response.
The accuracy and reliability of the information collected must be shown to be adequate for
the intended uses. For collections based on sampling, a special justification must be
provided if they will not yield "reliable" data that can be generalized to the universe
studied.
Action to Increase Response Rate:
Survey questions tailored to determine how well NOAA engages with constituents were
generated by a panel of social science experts, who met in a workshop and held multiple followup conference calls. The survey questions were developed using primarily a five-point Likert
scale rather than using all open-ended questions, to reduce response time and thus maximize the
response rate. This panel of experts addressed threats to content validity by developing a list of
questions that when answered match outcomes and conditions identified in a table of
specifications. The survey is relatively brief and has been carefully designed using input from
qualitative sessions and pre-survey testing of our key audience groups, so that respondent fatigue
is minimized. The survey questions will also be tested using statistical software, SPSS Statistics
18. NOAA Engagement staff will use Cronbach's alpha as a means to test the survey questions
reliability. Cronbach's alpha is a method of measuring a survey question’s reliability and internal
consistency. A high value, > 0.7, of alpha is often used as evidence that the items measure an
underlying construct. A 0.8 Cronbach alpha for question reliability will be considered
acceptable.
To minimize non-responses, respondents will be notified that they are invited to participate in a
NOAA Constituent Engagement Survey and that they do not have to supply their
names/addresses. The respondents will have the option to take the survey by using the online
version or a mailed hard copy. Respondents who are mailed a hard copy of the survey, will also
receive a prepaid postage return envelope. Surveys are distributed primarily electronically from
a secure location and non-biased third-party source. Distribution is computerized, so that all
contacts are automatically verified and classified. After the respondents have been notified a
search will be conducted to correct any email and postal mail address that are returned to the
third-party source.
Multiple respondents can connect simultaneously to the online survey. Any respondents who do
not appear to qualify for the survey have the option to bypass the survey. They will be thanked
for their time and routed to iGulf, NOAA’s Gulf of Mexico Regional Collaboration Team
2

website.
The non-respondents who originally received the survey online will be sent two follow-up email
reminders requesting that they complete the survey. A final follow up by telephone will be made
to non-respondents asking them to complete the online survey, offer to mail them a hard copy of
the survey, or to offer assistance in completing the survey questionnaire over the phone.
The non-respondents, who originally received a copy of the survey through the U.S. mail, will be
given one follow-up telephone call as a reminder. A final follow up by telephone will be made to
non-respondents asking them to complete the survey or to offer assistance in completing the
survey questionnaire over the phone.
Action to Address Non-Response:
If the response rate is below 80 percent, non-respondents will be contacted by phone to test for
non-response bias:
Fielded questions will include demographic elements such as:
•
•
•
•

What is your profession type?
How many years of experience do you have in your field?
What geographic area are you located in?
What is your degree of familiarity with NOAA products, services and information?

Other questions specific to the NOAA Constituent Engagement Survey that will be asked are:
•
•
•

What NOAA products, services and information did you use?
How likely are you to suggest NOAA’s products, services and information to others?
What was your overall level of satisfaction with NOAA?

This quick survey over the phone will help test the NOAA Constituent Engagement Survey for
non-response bias.
4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as
effective means to refine collections, but if ten or more test respondents are involved OMB
must give prior approval.
The survey was piloted by nine non-federal agents from the Engagement Working Group, which
consists of members from NOAA and Keystone Partner organizations (National Estuary
Programs, National Estuarine Research Reserve Systems, Sea Grant and Land Grant Universities
and Coastal Zone Management) with significant extension, outreach and education programs in
the Gulf of Mexico region. This pilot checked for functionality of the survey instrument, ease of
use and internal reliability. Based on feedback from the pilot respondents, minor revisions were
made to the survey.

3

5. Provide the name and telephone number of individuals consulted on the statistical
aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other
person(s) who will actually collect and/or analyze the information for the agency.
Statistics –LaDon Swann, Ph.D., Director of Mississippi-Alabama Sea Grant Consortium
(228) 818-8843
Eastern Research Group, Inc. - Aylin Sertkaya, Ph.D., Vice President
(781)674-7227 (Indefinite Duration, Indefinite Quantity contractor for NOAA)

4