2016-2018 Supporting Statement - Part B

Department of Commerce

United States Census Bureau

OMB Information Collection Request

Survey of State Government Research and Development

OMB Control Number 0607-0933

2. Collection of Information Employing Statistical Methods

1. Description of Universe and Respondent Selection

The survey frame consists of all state government-dependent units, including those for the District of Columbia and Puerto Rico, with the capacity or budget authority to perform or fund R&D. A state coordinator, who is appointed by the governor of each state, reviews the list of agencies and based on their knowledge of the state budget, activities, and priorities will make changes to the survey frame where appropriate.

The target population consists of state government departments, agencies, commissions, public authorities, institutions, and other entities that operate separately or somewhat autonomously from the central state government but where the state government maintains administrative or fiscal control over their activities, as defined in the U.S. Census Bureau Government Finance and Employment Classification Manual (chapter 1). Several industry-specific state commissions, which are generally chartered by state legislatures but are administered independently, are considered state agencies and included in the survey's population of interest. State-run colleges and universities, which are canvassed as part of NCSES's Higher Education Research and Development (HERD) Survey, are excluded from the survey frame. State-run laboratories or experiment stations controlled by state universities are also excluded from the respondent universe, as are any entities determined to be nonprofit or private, as defined by the U.S. Census Bureau government classification criteria.

Data are collected for each state’s fiscal year end. Most states have a fiscal-year period that begins July 1 and ends the following June 30. For example, FY 2016 is the state fiscal period beginning on July 1, 2015 and ending on June 30, 2016. There are, however, four exception to the June 30 fiscal-year end: New York (ends March 31), Texas (ends August 31), Alabama (ends September 30), and Michigan (ends September 30). For comparability, these four states are surveyed with the other 46 states that end on June 30 – a practice consistent with the Census Bureau’s Census of Governments program. The District of Columbia follows the Federal Government fiscal year, which ends September 30, while Puerto Rico’s fiscal year begins July 1 and ends June 30.

2. Procedures for Collection of Information

In October, a letter is sent to the office of each state’s governor asking them to appoint a state coordinator who would provide updates to the universe of state government-dependent agencies that have the capacity to perform or fund R&D during the most recent fiscal period ended. State coordinators are provided with a list of agencies in a Visual Basic Excel program (Attachment A) and asked to: add agencies that may be involved in R&D and are not already on the frame, make changes to the existing frame due to reorganizations, restructuring of agency R&D related activities to their portfolios since the last survey cycle, or remove agencies from the frame that no longer exist or have R&D activities.

Once the state coordinator completes updates to the list of agencies and returns it to the Census Bureau, introductory e-mails are sent to the agency contacts providing them with information about the survey and log-in credentials for the on-line survey instrument. State government agency respondents log in to a secure Internet-based survey instrument, Centurion, and complete the questionnaire. Upon completion by all agencies, the state coordinators are provided with an Excel file of agency responses and asked to review the final data for the state aggregate and agency level detail to identify any anomalies. Results are then provided to NCSES for final analysis and dissemination.

3. Methods to Maximize Response and Account for Nonresponse

Maximizing response: The key to maximizing response is obtaining high-level approval and oversight for the data collection effort within each state. The state coordinator, who has been identified by the Governor’s office, is asked to assist with identifying those agencies within the state that perform or fund R&D activities in a given fiscal year.

Survey staff monitors responses and works with the agency level respondents to obtain responses. Since the survey is expected to launch in November 2016, monthly follow-up on state agencies that have not submitted data will begin with state coordinators in January 2017.

Accounting for Nonresponse: Missing data are possible in two instances: (1) agencies within a state do not respond to the survey and, (2) agencies within a state do not respond to particular items on the survey. Since each state government’s organizational structure, laws, and delegation of powers within its purview are unique, there is no imputation base for a given state government to use data from other state governments. Therefore, there are no formal methods of imputation to account for these structural differences that are consistent with basic statistical methods. This practice is consistent with the Census Bureau’s Annual Survey of State Government Finances.

For the FY 2012 and FY 2013 survey collection, all state governments, the District of Columbia, and Puerto Rico participated in the survey; however only 45 out of 52, or 86.5 percent, of state coordinators officially verified final state aggregate data. The states of Georgia, Louisiana, New Mexico, Tennessee, Texas, Virginia, and the District of Columbia did not verify data officially, but instead did have some or all agencies submit data. The final agency-level response was 365 out of 366 agencies, or 99.7 percent, previously identified by state coordinators as subject to the survey. Of those agencies that responded 287, or 78.6 percent, reported having some R&D activity in FY 2012 or FY 2013. Response rates are expected to remain at similar levels for future iterations of the survey.

4. Tests of Procedures or Methods

Project staff consulted with employees from multiple state governments who hold roles primarily in the budget and finance offices, to test the collection methodology prior to the start of the FY 2014 and FY 2015 survey cycle. Usability testing on the Internet collection instrument (Centurion) was also conducted prior to the FY 2014 and FY 2015 survey to ensure the survey definitions were clear, the survey instrument was functional, and the survey was not overly burdensome for state coordinators or agency respondents. Protocols were developed in cooperation with the survey methodologists at both the Census Bureau and NCSES. Respondent debriefings will be conducted following the FY 2014 and FY 2015 survey cycle.

5. Contacts for Statistical Aspects and Data Collection

Statistical staffs at both the Census Bureau and the National Science Foundation have been consulted on statistical aspects of the survey and on data collection issues.

Persons responsible for cognitive interviews and usability testing:

Amy E. Anderson Riemer, Chief

Data Collection Methodology & Research Branch

Economic Statistical Methods Division

U.S. Census Bureau

(301) 763-7544

amy.e.anderson.riemer@census.gov

Rebecca L. Morrison, Survey Methodologist

Office of the Director

National Center for Science and Engineering Statistics

National Science Foundation

(703) 292-7794
(703) 292-9091 (FAX)

rlmorris@nsf.gov

Persons responsible for data collection:

Richard S. Hough, Assistant Division Chief

R&D and Special Surveys

Economic Reimbursable Surveys Division

U.S. Census Bureau

(301) 763-4823

(301) 763-4718 (FAX)

richard.s.hough@census.gov

Michael Flaherty, Chief

Research, Development, and Innovation Surveys Branch

Economic Reimbursable Surveys Division

U.S. Census Bureau

(301) 763-7699

(301) 763-4718 (FAX)

michael.j.flaherty@census.gov

Vicki Mills, Survey Statistician

Economic Reimbursable Surveys Division

U.S. Census Bureau

(301) 763-7678

(301) 763-4718 (FAX)

vicki.mills@census.gov

Persons responsible for analysis of the statistics and publication:

Christopher Pece, Senior Analyst

Research and Development Statistics Program

National Center for Science and Engineering Statistics

National Science Foundation

(703) 292-7788
(703) 292-9092 (FAX)
cpece@nsf.gov

John E. Jankowski, Program Director

Research and Development Statistics Program

National Center for Science and Engineering Statistics

National Science Foundation

(703) 292-7781

(703) 292-9091 (FAX)

jjankows@nsf.gov

List of Attachments

1. Survey Frame-Review Module (State Coordinator)

2. Centurion Collection Instrument Screenshots (Agency Respondent)

3. FY 2016 Survey of State Government R&D Reference Copy Questionnaire

4. Governor’s Letter to States

5. State Coordinator Cover Email

6. Agency Respondent Cover Email

7. Business Help Site Screenshot

8. Comments to Federal Register Notice

9. List of Individual Cognitive and Usability Testing Consultations

6