3145-0062 GSS Supporting Statement Part A

3145-0062 GSS Supporting Statement Part A.pdf

Survey of Graduate Students and Postdoctorates in Science and Engineering

OMB: 3145-0062

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 3145-0062 can be found here:

Document [pdf]

Download: pdf | pdf

2014−2016 Survey of Graduate Students
and Postdoctorates
in Science and Engineering
OMB Supporting Statement

July 2014

TABLE OF CONTENTS
Section

Page

A. Justification ........................................................................................................................... 3
A.1

Need for Data Collection and Legislative Authorization ........................................... 4

A.2

How, by Whom, and for What Purpose the Information Is to Be Used ..................... 5
A.2.1
A.2.2
A.2.3
A.2.4
A.2.5

Federal Uses ................................................................................................... 5
Use by Academic Institutions......................................................................... 6
Use by the Carnegie Foundation .................................................................... 6
Professional Societies Uses ............................................................................ 7
Media Uses ................................................................................................... 7

A.3

Consideration of Using Improved Technology .......................................................... 7

A.4

Identification of Duplication ...................................................................................... 8

A.5

Small Businesses Involvement ................................................................................. 11

A.6

Consequences of Less Frequent Surveying .............................................................. 11

A.7

Special Circumstances .............................................................................................. 12

A.8

Federal Register Notice and Consultations with Persons outside the Agency ......... 12

A.9

Payment or Gifts to Nonrespondents........................................................................ 12

A.10 Assurance of Confidentiality .................................................................................... 12
A.11 Sensitive Questions .................................................................................................. 12
A.12 Estimated Response Burden ..................................................................................... 13
A.13 Cost to Respondents ................................................................................................. 15
A.14 Cost to the Federal Government ............................................................................... 15
A.15 Changes in Burden ................................................................................................... 15
A.16 Project Schedule for Information Collection and Publication.................................. 15
A.17 Displaying the OMB Expiration Date ...................................................................... 16
A.18 Exceptions in Item 19 on Form 83-1 ........................................................................ 16
B. Collection of Information Employing Statistical Methods ................................................. 17
B.1

Respondent Universe and Sampling Procedure ....................................................... 17
B.1.1 Discussion of Institutional Frame................................................................. 17
B.1.2 NCSES Taxonomy of Disciplines and Changes to GSS Eligible Fields ..... 19

B.2

Description of Survey Methodology and Statistical Procedures .............................. 19
B.2.1 Imputation for Item Nonresponse in the GSS .............................................. 21

B.3

Methods Used To Maximize Response Rate ........................................................... 23

B.4

Testing of Procedures ............................................................................................... 25
B.4.1 Examination of Impact of Methodological Changes to Observed
Trends in the GSS .......................................................................................... 25
B.4.2 Anticipated Methodological Research ........................................................... 27
B.4.3 Changes to the 2014 GSS .............................................................................. 28

B.5

Names and Telephone Numbers of Individuals Consulted ...................................... 28
EXHIBITS

Number
Exhibit 1.
Exhibit 2.
Exhibit 3.
Exhibit 4.
Exhibit 5.
Exhibit 6.
Exhibit 7.
Exhibit 8.
Exhibit 9.
Exhibit 10.
Exhibit 11.

Page
Burden Estimates for the 2014 GSS ................................................................ 13
Burden Estimates for the 2015 GSS ................................................................ 14
Burden Estimates for the 2016 GSS ................................................................ 14
Total Burden Estimates for 2014–16 GSS ...................................................... 14
Annual GSS Survey Federal Government Estimated Costs ............................ 15
Number of Core and NewInstitutions, 2011-2012 .......................................... 17
Imputation Methods Used by Condition for 2012 GSS .................................. 21
Imputation Methods for 2012 GSS Key Totals, Counts and ..............................
Percentage of Total Cases ............................................................................... 23
Institution, School, and Unit Response Rates: 2010–12 ................................. 24
Consistency of NFR, Postdoc, and FT Grad Student Counts .......................... 27
Individuals Consulted on GSS Technical and Statistical Issues ..................... 28
LIST OF ATTACHMENTS

Attachment 1.

GSS Worksheet

Attachment 2.

GSS Web Survey Screen Shots

Attachment 3.

Federally Funded Research and Development Centers (FFRDCs) Postdoc
Survey Screen Shots

Attachment 4.

GSS Schedule

Attachment 5.

GSS Code List—Complete List of GSS Eligible Fields and Codes

Attachment 6.

NCES CIP and GSS Code Crosswalk

Attachment 7.

GSS Data Collection Plan

Attachment 8.

Improvements to the GSS Instrument

Attachment 9.

Summary of Changes Proposed for 2014 GSS

Attachment 10. First Federal Register Announcement

JUSTIFICATION
This submission requests a 3 year reinstatement of the previously approved OMB

clearance for the National Science Foundation’s (NSF’s) and National Institutes of Health’s
(NIH’s) Survey of Graduate Students and Postdoctorates in Science and Engineering (GSS). The
GSS is an annual survey that was last conducted in fall 2013. The OMB clearance for the GSS
will expire on October 31, 2014. With this clearance package, NSF requests approval to collect
data for the 2014–16 survey cycles.
The GSS is the only annual national survey that collects information on the characteristics
of graduate science, engineering, and health (SEH) enrollment for specific disciplines at the
departmental level. It also collects information for graduate enrollment on race and ethnicity,
citizenship, sex, sources of support, and type of support; information on postdoctorates (postdocs)
by citizenship, sex, sources of support, and type and origin of doctoral degree; and information on
other doctorate-holding nonfaculty researchers (NFRs) (see Attachment 2 for screenshots of the
GSS instrument). The GSS has been conducted by the NSF’s National Center for Science and
Engineering Statistics (NCSES) annually since 1972. Additional financial support is provided for
the GSS by the NIH.
The GSS is a census of all units (departments, programs, research centers, and health care
facilities) in science, engineering, and selected health fields within eligible academic institutions
in the United States with postbaccalaureate degree programs. The study collects aggregate
information on graduate students enrolled in these units, as well as postdocs and NFRs working
within these institutions. To improve coverage of postdocs, the GSS also periodically surveys
Federally Funded Research and Development Centers (FFRDCs) to gather information on the
race/ethnicity, sex, citizenship, source of support, area of research, and counts of the postdoctoral
appointments (see Attachment 3 for screenshots of the 2013 FFRDC survey instrument).
The GSS consists of two parts. In Part 1, the School Coordinator (SC) updates a list of all
eligible units in the school and classifies each unit by its GSS code (field). For established GSS
schools, this activity involves verifying the eligibility of units pre-populated from the previous
survey round, confirming GSS codes, adding any newly eligible units, and deleting defunct units.
All Part 1 activities are completed by the SC. In Part 2, data for each unit are entered or uploaded

by the SC or by designated unit respondents (URs), whom the SC may assign as needed. Part 2
requests details about graduate students, postdocs, and NFRs in each GSS-eligible unit.
Since April 2002, NCSES has been conducting extensive research and methodological
testing in GSS to reduce the respondent burden, improve data quality, reduce survey costs, and
improve processes that will result in a quicker release of the data to the public. The 2013 GSS,
and proposed 2014 GSS, reflect changes made to date based on the results of the research and
testing.
A.1

Need for Data Collection and Legislative Authorization
The National Center for Science and Engineering Statistics (NCSES) within the National

Science Foundation (NSF) collects, maintains, and disseminates information on science and
engineering resources in the United States. Specifically, Section 505 within the America
COMPETES Reauthorization Act of 2010 directs NSF as follows:
“(a) Establishment- There is established within the Foundation a National Center for Science
and Engineering Statistics that shall serve as a central Federal clearinghouse for the collection,
interpretation, analysis, and dissemination of objective data on science, engineering,
technology, and research and development.
(b) Duties- In carrying out subsection (a) of this section, the Director, acting through the
Center shall-(1) collect, acquire, analyze, report, and disseminate statistical data related to the
science and engineering enterprise in the United States and other nations that is
relevant and useful to practitioners, researchers, policymakers, and the public,
including statistical data on-(A) research and development trends;…”
The GSS provides a critical piece of the Foundation’s information that is used to meet its
responsibilities under the Act.

A.2

How, by Whom, and for What Purpose the Information Is to Be Used

A.2.1 Federal Uses
Information on the number and characteristics of students currently enrolled in graduate
SEH programs and of persons engaged in postdoctoral programs is extensively used by NSF and
NIH to assess future stock of trained SEH personnel. A variety of more general information needs
are met through the annual release of data in electronic format. NSF publishes a short InfoBrief
and a set of detailed statistical tables in the online report, Graduate Students and Postdoctorates
in Science and Engineering, available on the NCSES website.
Data from the GSS are also available as public use files, and on the Web through the
WebCASPAR (Computer Aided Science Policy Analysis and Research) system
(https://ncsesdata.nsf.gov/webcaspar). WebCASPAR is an institution-based data system, and it
contains institutional and summary data from all NSF academic sector surveys for all institutions
offering graduate-level instruction and/or maintaining research and design (R&D) activity in SEH
fields. Other data included in this system are those compiled from the Department of Education’s
National Center for Education Statistics (NCES) Integrated Postsecondary Education Data
System (IPEDS) surveys of Completions, Fall Enrollment, and Finance, and the NSF Survey of
Earned Doctorates. Primary uses of the data include: to review changing enrollment levels to
assess the effects of NSF initiatives; to track student support patterns; and to analyze participation
in SEH fields by targeted groups for all disciplines or for selected disciplines and for selected
groups of institutions. Program officers check departmental and institutional records, including
data from the GSS and NCES IPEDS surveys, to determine department eligibility for NSF
programs targeted to special populations or instructional programs.
NCES is also considering adding GSS data to its Data Lab PowerStats system.
NSF Uses
Special tabulations from the GSS data constitute a key resource in meeting policy and
program information needs of the Foundation. Major examples of GSS data uses are in the
Foundation’s two congressionally mandated biennial reports, Science and Engineering Indicators
and Women, Minorities, and Persons with Disabilities in Science and Engineering.
The GSS is one of four NSF surveys whose micro data are combined into an integrated
database to produce the publication Academic Institutional Profiles. The other three surveys are

(1) the Survey of Earned Doctorates; (2) the Higher Education Research and Development
(HERD) Survey; and (3) the Survey of Federal Science and Engineering Support to Universities,
Colleges, and Nonprofit Institutions. As explained in the next section, these data are further
integrated with institutional data from other NSF surveys and with surveys conducted by the
Department of Education. Together these data provide policy makers with information on the role
of higher education in the context of the national R&D effort.
Other Federal Uses
Data derived from the GSS are routinely provided to Congress and to various agencies of
the Executive Branch. Data have been used recently in the following ways:
•

Data on graduate SEH enrollment are provided annually to the Department of
Education’s NCES for comparison purposes and are published in the Digest of
Education Statistics.

•

Data in specially prepared GSS tabulations are used by the NIH to answer specific
questions to help their agencies prepare budgets and conduct program evaluation
studies.

A.2.2 Use by Academic Institutions
The surveyed institutions themselves are major users of the GSS data. Requests for the
data are received from numerous individual institutions and from national academic
organizations. The NCSES has been cooperating with the Association of American Universities’
Data Exchange (AAUDE) Group to provide them with comprehensive GSS data. Institutions use
the NSF’s GSS data reports or the WebCASPAR system to study selected groups of peer
institutions for planning and comparative purposes. They combine the NSF data with information
from state and local governments on institutions in their geographic areas. Institutions also use the
comparative data to review the strength of their own programs on the basis of factors such as
support of students by various federal agencies and progress in reaching special target
populations.
A.2.3 Use by the Carnegie Foundation
Data from the GSS are used by The Carnegie Foundation for the Advancement of
Teaching in developing the Carnegie Classification of Institutions of Higher Education. The
foundation uses the GSS data on nonfaculty research staff with doctorates as one component of

the “research activity” measure constructed for doctorate-granting universities (for more detail
see: http://classifications.carnegiefoundation.org/methodology/basic.php/).
A.2.4 Professional Societies Uses
Representative data users in this category include American Association of Colleges of
Nursing, American Association of Universities, American Chemical Society, American Council
of Education, American Geological Society, American Institute of Physics, American Society for
Engineering Education, Association of American Medical Colleges, Commission on
Professionals in Science and Technology, Computing Research Association, Council of Graduate
Schools, Federation of American Societies for Experimental Biology, and the National
Postdoctoral Association. Associations use GSS data to monitor trends in enrollment by field of
study, and many are also interested in tracking the numbers of postdoc and NFRs.
A.2.5 Media Uses
Enrollment of graduate students in science and engineering fields, particularly those
holding temporary visas, are well reported by the press, including the New York Times, the
Chronicle of Higher Education and other major national newspapers. A recent example of the
use of GSS postdoc data is a New York Times article on December 22, 2013, entitled “PartTimers Crowd Academic Hiring”.
A.3

Consideration of Using Improved Technology
Since the fall 1999 survey, GSS respondents have had the option to submit data by either a

paper form or through the website. During the past three cycles, all academic institutions have
reported electronically. The majority of respondents have welcomed the Web version of the GSS
for ease of submission and error resolution capabilities. Screenshots of the 2013 GSS Web
instrument are included in Attachment 2.
Reporting burden is stable or potentially reduced when the survey forms and questions are
stable and do not vary from year to year. Most of the academic institutions have been in the GSS
for many years, and some have established automated systems for assembling the requested data.
Most of the data that GSS collects are required by the academic institutions themselves for other
reporting requirements and for planning and evaluation purposes.

The Web version of the survey has a real-time monitoring system, allowing NSF to
monitor data, response status, system problems, and comments from respondents. From the
perspective of the respondents, the Web version is more convenient and simplifies the survey data
reporting (e.g., by automatically checking totals). NSF benefits from the use of the Web version
by receiving better quality data more quickly.
Respondents to the GSS have the option of uploading a data file, rather than completing
the Web version of the questionnaire. In 2012, 47 schools supplied their data via file upload.
Guidelines for the construction of the file are provided to respondents within the Web instrument,
along with a document providing answers to frequently asked questions. A template to help
coordinators construct a cross-walk between their internal unit codes and the GSS fields is also
provided, so that they can use data that have been downloaded from their internal database
systems. Coordinators may provide all of the data or a subset of the data via the upload
function. Typically the coordinators create a comma separated value (CSV) file that is viewable
in Excel.
The web instrument contains a help system that provides specifications on the format for
each question and link are provided to these specifications from the file upload page. In addition,
respondents can download a properly formatted CSV file pre-filled with their school’s current
information. When the respondent has created the file, he or she can click on the “upload” button
and the data are automatically loaded into the Web instrument. If there are formatting issues with
the file being uploaded, an error summary is provided to the user. This summary indicates the
nature of the error, as well as the line in the uploaded file where the error exists. When the upload
is complete, the respondent is then returned to the “unit status” page where any missing data,
errors or inconsistencies in the data are displayed. After all information is completed and all
issues are resolved, the respondent can submit the data.
A.4

Identification of Duplication
NSF staff consults regularly with other federal agencies and private organizations to

prevent duplication of data collection activities and to stay abreast of changes in other surveys.
Such consultations take place with the Department of Education’s NCES, the Council of Graduate
Schools (CGS), and others. Specific surveys conducted by these groups will be discussed below.
In addition, staff of the NCSES participate in a variety of NCES-related activities, including
8

serving on the 2010 Classification of Instructional Programs (CIP) Working Group and Technical
Review Panels. The routine data uses of the federal agencies described in Section A.2 have
largely determined the content of the questionnaire.
Only the GSS collects the following information at the level of detailed SEH fields of
study:
•

•

–

For full-time graduate students, aggregate counts by
–

Sources of major financial support (federal agencies, institutions, self-support, etc.)

–

Mechanisms of major financial support (fellowships, teaching assistantships, etc.)

–

gender

–

citizenship

–

enrollment status (full-time or part-time; first time)

–

race/ethnicity background of U.S. citizens

For part-time graduate students, aggregate counts by
–

gender

–

citizenship

–

race/ethnicity background of U.S. citizens

For postdocs, aggregate counts by
–

sources of major financial support

–

Mechanism of major financial support

–

gender

–

citizenship

–

type of doctoral degree

–

doctoral degree origin

For NFRs, aggregate counts by
–

gender

–

type of doctoral degree

Because the data are collected from all eligible institutions with graduate SEH
departments, data are available at the detailed field of study by institutional characteristics, such
as highest degree granted, geographical location, type of control (public or private), or any other
special grouping (medical schools, historically black colleges and universities, land-grant

institutions, etc.) as well as by rankings on various characteristics (foreign enrollment, minority
enrollment, field-specific enrollment, etc.)
Some graduate enrollment data are collected by other organizations, either federal or
private, but none of the other data collection efforts contain the detailed field distribution that is
required for analyses and provides the necessary data for the NSF and NIH. Furthermore, no other
surveys collect data on federal agencies’ support of graduate students.
The IPEDS, for example, collects race and ethnicity data every 2 years for nine selected
fields, of which four are within the NSF definition of science and engineering (and at a more
general level than is collected for GSS). The IPEDS annual fall enrollment data collected by race
and ethnicity category are not reported by the field and hence do not provide a viable substitute
for the race and ethnicity data collected in the GSS. No data are collected on source of support or
on postdocs and NFRs. The categories used on the GSS are in compliance with the OMB
race/ethnicity guidelines.
The CGS conducts an annual survey of graduate enrollment in cooperation with the
Graduate Records Examinations (GRE) Board, surveying 787 institutions in 2012 that were
members of the CGS or one of the four regional graduate school associations—the Conference of
Southern Graduate Schools, the Midwestern Association of Graduate Schools, the Northeastern
Association of Graduate Schools, and the Western Association of Graduate Schools. The survey
had a response rate of 86%, with 675 schools responding. The survey collects data by fifty-one
fine fields of study using the GRE discipline codes as its taxonomy, type of institutional control,
and highest level of degree offered, but has no data on source of financial support. It also collects
information on postbaccalaureate and post-master’s certificates and applications to graduate
schools. Only the GSS maintains detailed data grouped into ninety-seven fine fields of study on
all SEH degree field at all eligible institutions and institution-provided data on source of financial
support.
A number of surveys are conducted by other professional societies or by groups of
institutions, and are limited to a single field or group of related fields or to institutions that are
members of the organization. These surveys may collect far more detailed data on the fields of
interest to the organization conducting the survey, and may even collect data on topics not
covered by the GSS (e.g., on undergraduate enrollment), but they do not provide compatible data

on all SEH fields, nor do they often address the issue of types and sources of financial support for
graduate students.
For the past several years, NCSES has conducted a study to determine the feasibility of
collecting data on the number and characteristics of postdocs in the United States. In 2009, NSF
piloted an expanded postdoc section within the GSS, and based on the results, implemented the
new section for all institutions in the 2010 cycle. For the foreseeable future the GSS will continue
to collect information on the postdocs, including the foreign postdocs, in the academic sector.
A.5

Small Businesses Involvement
The survey universe consists entirely of U.S. universities and colleges that enroll graduate

students and FFRDCs that have postdoctoral appointments.
A.6

Consequences of Less Frequent Surveying
A less frequent survey cycle would have several serious consequences. The first, of

course, would be the loss of information. Because of the data uses described previously, biennial
or less frequent surveys means that data users would be unable to access what is now current
information. Minor shifts in enrollment trends are monitored as early indicators of likely future
changes in the supply of SEH professionals.
Collecting the GSS annually increases the value of the data for monitoring trends,
particularly the effects of dramatic changes in the larger context. Recent examples are changes in
enrollment that correspond to the dot-com boom and bust, the events of September 11, 2001, and
the economic downturn in 2008. Less than annual data collection may not capture such changes
or reveal the inflection point of a changing trend. For the past few years, the release of the GSS
fall enrollment data has been eagerly awaited to see the trends in SEH graduate enrollment in
foreign visa holders post 9/11. Enrollment for this pool of graduate students did not drop
immediately (i.e., in 2001), and the trends differed by several years for first-time enrollment and
total enrollment. Those nuances would have been lost if the data had not been collected every
year.
Most colleges and universities have automated record keeping systems, facilitating their
ability to respond to the GSS on an annual cycle. These automated record systems considerably
reduce the time required to assemble and report information needed for the GSS related to
11

graduate enrollment by field, postdoctoral appointments, and sources and mechanisms of support,
etc. Thus, because the database and software are retained, kept current, and easily accessed,
collecting consistent data annually considerably reduces respondent burden for academic
institutions with automated data systems.
Annual collection also contributes to the continuity of contacts with the SCs within
institutions. Having this continuity helps the SCs maintain their databases and, therefore, maintain
the quality of the data.
A.7

Special Circumstances
There are no special circumstances.

A.8

Federal Register Notice and Consultations with Persons outside the Agency
The Federal Register notice was published on May 21, 2014 (see Attachment 10).
NSF regularly consults with the Department of Education’s NCES and other federal

agencies, such as NIH, professional societies, and university staff. NSF staff members maintain
frequent contact with members of the data-using community as well as with major academic data
providers through attendance at professional society meetings and consultation with institutional
and agency officials. GSS sessions are typically held at the Association for Institutional
Researchers (AIR) Annual Forum each year to obtain respondent input.
A.9

Payment or Gifts to Nonrespondents
Not applicable. There are no payments to GSS respondents.

A.10

Assurance of Confidentiality
No pledge of confidentiality is given to institutions providing data to the GSS. Data

collected in the GSS are aggregate counts of students, postdocs, and NFRs. Data are published
only at the departmental summary level.
A.11

Sensitive Questions
The survey questionnaire does not contain any questions of a sensitive nature.

A.12

Estimated Response Burden
Each year, when respondents reach the end of the web instrument, they are asked to report

how long it took them to complete the GSS. The average burden per unit in GSS decreased from
2.8 hours in 2011 to 2.6 hours in the 2012 GSS. In the last OMB clearance request, we had
estimated 2.7 hours in 2011 and 2.6 hours in 2012, which is very close to the reported burden
hours. We had also estimated a slightly higher number of units for the survey so the total reported
burden hours for those years were lower than the burden hours approved by OMB.
In keeping with prior experience, we estimate that the per unit burden will be stable or
decrease slightly over time as respondents become familiar with the items in the survey, thus we
estimate a burden of 2.6 hours per unit in 2014. We anticipate that the number of units reporting
in 2014 cycle will include fewer units than 14,429 units in 2013 due to deletion of 655 units that
will no longer be eligible based on the new NCSES Taxonomy of Disciplines (see Section B.1.2),
plus approximately a 1% increase. A total of 36,171 burden hours are requested for the 2014
cycle of GSS. These figures are presented in Exhibit 1.
Exhibit 1. Burden Estimates for the 2014 GSS
Category
2013 existing units less 655 units
deleted due to new NCSES Taxonomy
of Disciplines

Respondents
(# of units)

Burden/unit
(hours)

Total burden
(hours)

13,774

2.6

35,812

138

2.6

359

1% increase from 2013
Estimated total

13,912

36,171

For the 2015 GSS, the total respondent burden is estimated at 35,268 hours and the burden
for 2016 will be 35,481 hours, as shown in Exhibits 2 and 3. The 2015 GSS burden hours also
includes the hours for the Survey of the Postdocs at FFRDCs. This survey was last conducted in
2013 and is currently scheduled to be conducted bi-annually.

Exhibit 2. Burden Estimates for the 2015 GSS
Category
2014 existing units

Respondents
(# of units)

Burden/unit
(hours)

Total burden
(hours)

13,912

2.5

34,780

139

2.5

348

3.5

140

1% increase from 2014
Survey of Postdocs at FFRDCs
Estimated total

14,091

35,268

Exhibit 3. Burden Estimates for the 2016 GSS
Category
2015 existing units

Respondents
(# of units)

Burden/unit
(hours)

Total burden
(hours)

14,051

2.5

35,128

141

2.5

353

1% increase from 2015
Estimated total

14,192

35,481

In addition, NCSES is requesting 360 burden hours over 3 years for future methodological
testing needs. Exhibit 4 summarizes the burden estimates for the next 3 years of the GSS. The
estimated average annual burden across the 3 cycles is 2.54 hours per unit.
Exhibit 4. Total Burden Estimates for 2014–16 GSS
Respondents
(# of units)

File Type	application/pdf
Author	RTI_DP
File Modified	2014-08-01
File Created	2014-08-01