Supplemental and Part B questions

NOAA’s Technology Partnership Office SBIR and CRADA Survey

February 28, 2018

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

This request is for a public survey to be conducted by NOAA’s Technology Partnerships Office (TPO).

The TPO oversees two programs that help foster product development and technology innovation and transfer: NOAA’s Small Business Innovation Research (SBIR) program and the Technology Transfer Program. The SBIR program encourages small business innovation by providing federal research and development (R&D) funding critical to small businesses that develop products with the high probability of commercialization and align with NOAA’s mission.^1,2,3 NOAA’s Technology Transfer Program encourages the collaboration and transfer of information through the use of Cooperative Research and Development Agreements (CRADAs), which can speed the commercialization of NOAA-developed technology and provide other collaborator benefits, such as protecting research results from public disclosure for a five-year period of time. ⁴ While these programs certainly generate social and economic benefits for participants, product users, and the broader economy of the United States, these benefits have not yet been described and quantified.

Cooperative Research and Development Agreements (CRADA)

CRADAs help facilitate the sharing and commercialization of products and innovations developed in NOAA’s laboratories and science centers throughout the country. A CRADA allows NOAA and non-Federal partners to optimize their resources, share technical expertise in a protected environment, share intellectual property emerging from the effort, and speed the commercialization of NOAA developed technology by:

• Providing incentives that help speed up the commercialization of NOAA developed technology.

• Protecting propriety information brought to the CRADA effort by the partner.

• Allowing all parties to keep research results emerging from the CRADA confidential and free from disclosure through the Freedom of Information Act for up to five years.

• Allowing NOAA and the partner to share patents and patent licenses.

Small Business Innovation Research (SBIR)

The SBIR program facilitates the involvement of small businesses in federal research/R&D that has the potential for commercialization through a competitive, awards-based program that provides funding to awardees.¹ There are four goals of the program¹:

• Stimulate technological innovation

• Meet Federal research and development needs

• Foster and encourage participation in innovation and entrepreneurship by women and socially or economically disadvantaged persons

• Increase private-sector commercialization of innovations derived from Federal research and development funding

NOAA’s TPO is interested in the extent to which the two programs are meeting the needs of project partners, their satisfaction with the programs, and the results that are being generated by the programs in terms of meeting the overall program objectives.

This request covers two surveys, one for each program, to be implemented by TPO. In these surveys, TPO will ask about:

• Satisfaction with the program

• The extent to which the program is meeting partner needs

• Suggestions for improvement

• The types of outcomes that the companies have achieved because of the program (e.g., revenues, numbers of jobs created, etc.)

2. Explain how this survey was developed. With whom did you consult regarding content during the development of this survey? Statistics? What suggestions did you get about improving the survey?

TPO has contracted with Eastern Research Group, Inc. (ERG) to develop and implement a study that assesses the value of the CRADA and SBIR programs from the perspective of the companies that have partnered with NOAA in each program. ERG has developed several surveys for NOAA programs that are designed to assess how programs are working and to collect detail for NOAA on how to improve programs. TPO and ERG worked on the included survey instruments for several weeks and finally determined that these will provide the needed information to assess the program.

NOAA has not used statistical sampling in designing the survey; all program partners and/or collaborators will be surveyed. NOAA does not anticipate the use of statistics in analyzing the data; ERG will be primarily tabulating distributions across value and calculating means from the resulting data.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

How the Survey Will Be Conducted

NOAA TPO maintains contact information for all SBIR partners and CRADA collaborators. This contact information will be used to reach out to them to invite them to take an online survey. NOAA has selected an online instrument since it will be the most convenient form or survey for the companies that are involved in the two programs.

Response Rate

NOAA expects that response to this survey will be greater than 80 percent; the reason for this high estimate is that NOAA either works collaboratively with these companies (CRADA) or has provided funding to these companies to allow for product development. Thus, NOAA has either an existing or a recent relationship with these companies. NOAA expects that non-response would primarily be generated from out-of-date contact information and/or companies that have ceased operations.

Maximizing Response

To ensure a maximum response rate, the TPO contractor ERG will perform the following tasks:

• ERG has developed a survey that minimizes the burden on respondents by using good survey design. This includes developing well-written questions and limiting the number of the questions to the minimum necessary.

• ERG will use multiple prompts to generate responses. ERG will use a pre-notification email to respondents, an email that asks the respondent to take the survey, and then two reminder emails.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

The survey data will be analyzed by tabulating the data to characterize the distribution of responses across values for the questions that are asked, as well as calculating mean values for quantitative questions. NOAA may also have ERG cross-tabulate questions (e.g., levels of satisfaction by year in which the company received its SBIR grant, levels of satisfaction by industry category).

No statistical sampling techniques are being employed for this survey; NOAA is collecting data from the full population of SBIR partners and CRADA collaborators.

The data do not directly contribute to a GPRA measure.

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

Respondent Universe

TPO will send the survey to a set of SBIR partners and CRADA collaborators that cover most program participants. For SBIR, the set of companies included in the respondent universe cover SBIR agreements issued by NOAA from 2004-2017. These agreements and corresponding companies were selected based on having moved past the Phase I planning phase of SBIR and into the Phase II (prototyping and pre-commercialization) phase of the program. As such, there is more likely to be a measurable impact from their participation in the program. For CRADA, TPO selected a set of 26 CRADA agreements to include in the survey. These agreements were selected as they have either been completed or are in a later stage of activity. We believe CRADAs signed in the past 1-2 years are less likely to have generated measurable results than those that are in years 3-5 of an agreement or those where the collaboration has been completed. Table 1 summarizes the total number agreements within each program; this reflects the total number of surveys that will be sent.

Table 1: Populations and Sample Information

Program	Total Number of Agreements	Expected Response Rate	Expected Number of Completed Survey Responses
SBIR	132	80%	106
CRADA	25	80%	20

Selection Method

All SBIR and CRADA recipients are being included in the survey.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Stratification, Sample Size, and Precision and Accuracy

No statistical methods are being employed for this survey.

Unusual Problems Requiring Specialized Sampling Procedures

None are required.

Periodic Data Collection Cycles

This request is for a one-time data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Maximizing Response Rates

The TPO contractor, ERG, will employ the following strategies to maximize response rates from the sample:

• ERG has developed a survey that minimizes the burden on respondents by using good survey design (and is expected to take 20 minutes only). This includes developing well-written questions and limiting the number of the questions to the minimum necessary.

• ERG will use multiple prompts to generate responses. ERG will send a pre-notification email to respondents about the survey, an email that asks the respondent to take the survey, and then two reminder emails.

Adequacy for Intended Uses

The survey data are being used by TPO to better inform TPO management and staff on how well they are meeting the needs of program participants. The data will be useful for identifying ways in which the programs have led to positive outcomes for companies involved in each program. The data will also provide an overview of the outcomes and results that program participants have attained. The data from these surveys will be more than adequate to meet these needs.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

TPO’s contractor, ERG, will conduct nine or fewer interviews with SBIR and CRADA staff prior to fully implementing the survey. These interviews will allow ERG to make minor adjustments to survey wording to ensure that respondents understand what is being asked.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The TPO has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA, to design the survey instrument, develop the sampling approach, implement the survey, and analyze the resulting data collected. The survey design team included the following individual:

Dr. Lou Nadeau (781) 674-7316; lou.nadeau@erg.com

1 SBIR program webpage: https://www.sbir.gov/about/about-sbir#sbir-program

2 NOAA SBIR webpage: http://techpartnerships.noaa.gov/SBIR.aspx

3 There are three core elements to NOAA’s mission of “Science, Service and Stewardship: 1) understand and predict changes in climate, weather, oceans and coasts; 2) share that knowledge and information with others; and 3) conserve and manage coastal and marine ecosystems and resources.” http://www.noaa.gov/our-mission-and-vision

4 NOAA Technology transfer (http://techpartnerships.noaa.gov/TechTransfer.aspx) and CRADA (http://techpartnerships.noaa.gov/TechTransfer/CRADAs.aspx) webpages.

TPO SBIR and CRADA Satisfaction Survey 7