Changes to the SED 2018-19 Questionnaire from the SED 2015 Questionnaire

ATTACHMENT 2

CHANGES to the SED 2018 Questionnaire from the SED 2017 Questionnaire

Items Added to the Questionnaire

PART A2 – EDUCATIONAL HISTORY

New item inserted near the end of Part A2:

Rationale for change: Knowing which degrees were earned simultaneously will help in deciphering respondents’ educational history timelines.

PART B – POSTGRADUATION PLANS

Response option #2 (revised from 2016 SED) inserted into B2 of the paper questionnaire:

Rationale for change: This change was made to the 2017 web survey after the 2017 paper questionnaires were already printed. Therefore, this change to the 2018 paper questionnaire simply brings it in line with the web instrument.

Employed Other Than Postdoc or Training section

New item inserted near the beginning of the section:

Rationale for change: For respondents who report that their principal employer will be a U.S. 4-year college or university, a U.S. medical school, a U.S. university-affiliated research institute, a U.S. community or 2-year college, or a foreign educational institution, data users have indicated that it would be useful to know whether these individuals will be holding a tenure-track faculty position.

Two new items inserted at the end of the section:

Rationale for change: Among respondents who indicate they have accepted a position other than a postdoc or training, and that position (and employer) is the same they held before or during their doctoral studies, it is important to distinguish between (i) respondents holding positions they consider to be short-term jobs while they continue to search for a ‘permanent’ position, (ii) respondents who accepted what they consider to be a ‘permanent’ position before completing their doctoral studies, and (iii) respondents who are returning to their pre-doctoral positions.

New NEGOTIATING OR SEEKING section:

Rationale for change: In prior SED questionnaires, respondents who reported that they did not have a definite commitment for employment or training were skipped to the demographics section and no information was collected about their postgraduation plans. The 2018 SED questionnaire seeks to rectify this gap by requesting information about the type of position and employer these individuals are seeking.

Moreover, during cognitive testing it was found that individuals who have not yet found the type of position they are seeking, may currently be employed. Hence, a question pertaining to their current employment is also included.

Items Removed from the Questionnaire

PART A – EDUCATION

Rationale for change. This item is recommended for removal because it has relatively high burden on respondents and low usefulness, as it is not coded and reported in standard NCSES publications, nor is it included in the Doctorate Records File or released to researchers, because it is personally identifiable information.

Rationale for change. These items are recommended for removal because the resulting data have not been found to be useful to researchers and analysts due to the conceptual overlap of the items, the rounding of responses to whole years, and the lack of data variation across subgroups of respondents. The redesigned questionnaire originating with the 2017 SED measures time-to-degree more precisely by capturing the start and end dates of all of the degrees a respondent has earned, and so removing the four items would reduce the burden on respondents. When roughly 50 past SED data users were alerted to this proposed change, none of them disagreed.

Items Modified in the Questionnaire

PART C – BACKGROUND INFORMATION

Item C4 requests the highest educational attainment of respondents’ parents or guardians. In past versions of the SED questionnaire, respondents were asked to provide this information for each of their mother and father. In the 2018 instruments the labels will be changed from “Mother” and “Father” to “Mother/Female Guardian” and “Father/Male Guardian.” In addition, these labels will not be fixed in the web survey, but rather they each will be selected by respondents such that a respondent could report for two mothers/female guardians or two fathers/male guardians, rather than one of each.

Here is the revised version from the paper questionnaire:

Here is the current version from the web survey:

Here is a mock-up for the web instrument (the label “Mother/Father” will be changed):

Rationale for change. Respondent comments in previous cycles of the SED and in cognitive interviews conducted for the National Survey of College Graduates identified the need to broaden the question to encompass non-traditional families that may consist of same-sex parents rather than a mother and a father in the traditional sense.

Item C17 requests the contact information of a person who is likely to know where the respondent can be reached in the future. The text of the introduction will be modified to explain why this information is being requested. In addition, the label under the second text box will be changed from “Street Address” to “Relationship.” In the web instrument, if the relationship is personal (e.g., family member or friend), street address will be requested along with the other contact information. If the relationship is business-related, the company name will be requested along with the other contact information.

Here is the revised version from the paper questionnaire:

Rationale for change. This information is collected so that the respondents who are selected into the sample for the Survey of Doctorate Recipients (SDR), which is a separate NCSES survey, can be located in the future. The SDR’s locating team has advised that knowing the relationship of this contact person to the respondent is key information in their locating efforts. Moreover, street address is useful for personal contacts but not business contacts, for whom knowing their company name is more useful.

Confidentiality Pledge

The Confidentiality Pledge included in the 2017 SED instruments is as follows:

This information is solicited under the authority of the National Science Foundation Act of 1950, as amended. All information you provide is protected under the NSF Act and the Privacy Act of 1974, and will be used only for research or statistical purposes by your doctoral institution, the survey sponsors, their contractors, and collaborating researchers for the purpose of analyzing data, preparing scientific reports and articles, and selecting samples for a limited number of carefully defined follow-up studies. The last four digits of your Social Security Number are also solicited under the NSF Act of 1950, as amended; provision of it is voluntary. It will be kept confidential. It is used for quality control, to assure that we identify the correct persons, especially when data are used for statistical purposes in Federal program evaluation. Any information publicly released (such as statistical summaries) will be in a form that does not personally identify you or other respondents. Your response is voluntary and failure to provide some or all of the requested information will not in any way adversely affect you.

The time needed to complete this form varies according to individual circumstances, but the average time is estimated to be 20 minutes. If you have comments regarding this time estimate, you may write to the National Science Foundation, 4201 Wilson Blvd., Arlington, VA 22230, Attention: NSF Reports Clearance Officer. A Federal agency may not conduct or sponsor a collection of information unless it displays a currently valid OMB control number.

As a result of the implementation of EINSTEIN, the Confidentiality Pledge provided in the 2018 SED instruments will include an additional sentence as highlighted below:

This information is solicited under the authority of the National Science Foundation Act of 1950, as amended. All information you provide is protected under the NSF Act and the Privacy Act of 1974, and will be used only for research or statistical purposes by your doctoral institution, the survey sponsors, their contractors, and collaborating researchers for the purpose of analyzing data, preparing scientific reports and articles, and selecting samples for a limited number of carefully defined follow-up studies. The last four digits of your Social Security Number are also solicited under the NSF Act of 1950, as amended; provision of it is voluntary. It will be kept confidential. It is used for quality control, to assure that we identify the correct persons, especially when data are used for statistical purposes in Federal program evaluation. Any information publicly released (such as statistical summaries) will be in a form that does not personally identify you or other respondents. Per the Federal Cybersecurity Enhancement Act of 2015, your data are protected from cybersecurity risks through screening of the systems that transmit your data.  Your response is voluntary and failure to provide some or all of the requested information will not in any way adversely affect you.

The time needed to complete this form varies according to individual circumstances, but the average time is estimated to be 20 minutes. If you have comments regarding this time estimate, you may write to the National Science Foundation, 4201 Wilson Blvd., Arlington, VA 22230, Attention: NSF Reports Clearance Officer. A Federal agency may not conduct or sponsor a collection of information unless it displays a currently valid OMB control number.

FIELD OF STUDY TAXONOMY

Field of Study Review

Every two years the Survey of Earned Doctorates evaluates the taxonomy of fields of study used in the SED to consider fields for addition to or deletion from the taxonomy. The survey contractor provides analytical support for this review, including tabulations of responses to the field of study questionnaire items and content analyses of verbatim responses. Field of study responses not currently in the taxonomy but which appear in the verbatim responses of many respondents over the most recent three-year period are considered “possible emerging fields” and become candidates for addition to the SED taxonomy. Fields of study currently in the taxonomy that are not selected by many respondents over the most recent three-year period become candidates for deletion from the SED taxonomy. Additional information sources – including scans of university websites, reference to the Classification of Instructional Programs (CIP) taxonomy, and the Federal Sponsors’ subject-area expertise – are brought to bear on the decisions to add or delete particular fields of study (see table 1 below).

As a result of the latest (December 2016) field of study review by the Federal Sponsors, the following three fields of study have been added to the SED taxonomy:

Description of field of study added	Field of study code
African American Studies, Literature, and History	775
Computational Mathematics	461
Learning Sciences	801

Two fields have had their labels modified:

Old label	Revised label	Field of study code
Drama/Theater Arts	Drama/Theater Arts /Performance Studies	795
Film/Cinema/Video Studies	Film/Cinema/Media Studies	778

One field has been dropped from the SED taxonomy:

Description of field of study deleted	Field of study code
Wood Science & Pulp/Paper Technology	072

Table 1. Fields to consider for addition to or deletion from the SED Field of Study taxonomy on the 2018 and 2019 questionnaires

SED emerging field verbatim	Proposed field label	Explanation of proposed change	Final recommendation
African American Studies, Literature, and History	African-American/Black Studies	Proposed field label matches CIP field label and the proposed SED field code falls within the Humanities: Other Humanities. In addition, African-American Studies has doctoral programs, professional associations, and publications that are distinct from other cultural studies. NSF: Proposed location of this field (in Humanities) conflicts with the Taxonomy of Disciplines (which locates it in Social Sciences, as a subfield of Area/Ethnic Studies); however, most respondents with this verbatim selected a field code in Humanities (26 of 42 respondents, according to data from 2016 Emerging Fields analysis). Recommend either: (i) locating this field in Humanities with the field label "African American Studies, Literature, and History," or (ii) locating this field in Social Sciences with the proposed field label.	Add field as 775 African American Studies, Literature, and History in the Humanities.
Computational Mathematics	Computational Mathematics	The proposed SED field label Computational Mathematics matches the CIP field label. Computational Mathematics has doctoral programs, associated academic journals, and professional associations. In addition, Computational Mathematics merits its own funding program within the National Science Foundation Division of Mathematical Sciences.	Add field as 461 Computational Mathematics
Learning Sciences	Learning Sciences	The proposed field label matches the CIP field label, and the proposed SED field code falls within the Education: Research & Administration FOS family title, as the CIP field area is Educational Assessment, Evaluation, and Research. Furthermore, Learning Sciences has doctoral programs, professional associations, and academic journals.	Add field as 801 Learning Sciences
Performance Studies	Drama/Theater Arts /Performance Studies	No exact match in CIP, but Performance Studies does include doctoral programs, professional associations, and publications. There is no indication that Performance Studies is distinct from Drama/Theater Arts; therefore, we recommend expanding that label to include Performance Studies.	Change label to Drama/Theater Arts/Performance Studies
Film and Media Studies/ Cinema and Media Studies	Film/Cinema/Media Studies	The review of verbatim strings revealed numerous "Film and Media Studies" and "Cinema and Media Studies" research doctorate fields were reported. The SED taxonomy currently includes field 778 Film/Cinema/Video Studies. We recommend adding "Media" to the label, possibly replacing the word "Video," which appears in relatively few strings.	Change label to Film/Cinema/Media Studies

Table 1. Fields to consider (continued)

SED emerging field verbatim	Proposed field label	Explanation of proposed change	Final recommendation
Behavioral Neuroscience	Behavioral Neuroscience	The proposed field label closely matches the CIP field label "Neurobiology and Behavior." The proposed SED field code falls within the Life Sciences: Biology/Biomedical Sciences. And, research indicates that Behavioral Neuroscience includes doctoral programs, and professional associations and academic journals that are distinct from Neuroscience. If this field is added, it is recommended that it is cross-referenced with SED FOS 626 Cognitive Neurosciences on the questionnaire. NSF: The potential overlap with Cognitive Neurosciences (code = 626) is a concern, especially as that field did not appear on the SED until 2016. Field codes of respondents with the Behavioral Neuroscience and Cognitive Neuroscience verbatims (according to the 2014 Emerging Fields memo) were both split between Social Sciences (600's) and Biological Sciences (100's) -- Cognitive Neuroscience had more SocSci than BioSci, and Behavioral Neuroscience had more BioSci than SocSci (data from 2016 Emerging Fields analysis showed this BioSci tendency for Behavioral Neuroscience even more strongly than the 2014 analysis).	Do not add field now, but reconsider based on next review.
Biological systems/ Biosystems Engineering	Biological and Biosystems Engineering	Proposed field label matches CIP field label, and falls within the CIP broad field of "ENGINEERING," and it is recommended that the proposed field be placed in the FOS family of "Engineering." The proposed field of Biological and Biosystems Engineering is distinct from SED FOS 306 – Bioengineering and Biomedical Engineering, in that it has specific doctoral programs, and academic journals.	Do not add field now, but reconsider based on next review.
Sustainability Education		No exact match in CIP. Sustainability Education has doctoral programs, and distinct professional associations and publications. However, only three institutions granted research doctorates in each of the three years examined. NSF: Field codes of respondents with this verbatim were drawn from several broad fields: 3 from AgSci/Natural Resources, 2 from Engineering, 14 from SocSci, 7 from Education, and 10 from Other Fields Not Elsewhere Classified.	Do not add new field
		Drop field 072 Wood Science & Pulp/Paper Technology	Drop field

NCSES Taxonomy of Disciplines (TOD)

In addition to the changes resulting from the field of study review, a few other modifications were made to the SED taxonomy as a result of implementing the NCSES Taxonomy of Disciplines (TOD), meant to provide more consistency across NCSES surveys. To be compliant with the TOD, the following revisions were made to the field of study list that appears within the SED paper questionnaire.

Field	Old Taxonomy Location	New Taxonomy Location
Natural Resource, Environmental Policy	Social Sciences	Life Sciences: Agricultural Sciences & Natural Resources
Medical Physics, Radiological Science	Physical Sciences: Physics	Life Sciences: Health Sciences
Marine Biology & Biological Oceanography	Life Sciences: Biological & Biomedical Sciences	Physical Sciences: Ocean & Marine Sciences
American, U.S. Studies	Humanities: Other Humanities	Social Sciences
Health Policy Analysis	Life Sciences: Health Sciences	Social Sciences
Robotics	Computer & Information Sciences	Engineering
Theology, Religious Education	Fields Not Elsewhere Classified	Humanities: Other Humanities
Archaeology*	Humanities: Other Humanities	Humanities: Other Humanities; Social Sciences

*Archaeology is cross-listed in the SED field of studies list, appearing in both the Humanities and Social Sciences sections.