SED Methodological Research Listing

ATTACHMENT 9.1

Methodological Research Concerning the Survey of Earned Doctorates
Note: Reports available upon request from Mark Fiegener (mfiegene@nsf.gov).

Results From the Survey of Earned Doctorates Web Survey of SED Data Users: Key
findings from a web survey of SED data users on their reactions to small cell data suppression.
(Simko, Hess, and Dominguez, 2008).
Results From the Survey of Earned Doctorates Cognitive Interviews: Key findings and
recommendations from a set of 20 cognitive interviews conducted on the SED with emphasis on
the reaction to new formatting. (Simko, Hess, and Dominguez, 2008).
Survey of Earned Doctorates: Results from the Survey of Earned Doctorates Scanning
Experiment: Findings and recommendations from the optical scanning experiment conducted
during AY 2007. (Webber, Lashley, Ahsan, 2008)
Survey of Earned Doctorates: Report on the Impact of the Introduction of a four-digit SSN
item on the SED: A report on the methodology and analysis of the four-digit SSN experiment
conducted in AY 2006. (Welch, Groenhout, Webber, Williams, 2008)
Summary of Respondent Comments on the Survey of Earned Doctorates: 2001- 2007:
qualitative analysis of respondent verbatim comments spanning 2001-2007 survey forms.
(Lashley, Hess, and Reyes, 2008)
Survey of Earned Doctorates: Evaluation of the Addition of Questions on Expected Salary
to the Survey of Earned Doctorates in 2008: Comprehensive findings and recommendations
from the various interviews, focus groups, validations tests, and quantitative experiments
conducted to assess the addition of a salary question to the 2008 questionnaire. (Latter, Hess,
Webber, 2007)
SED Research Doctorate Types: Procedures Followed in the NORC Review of Research
Doctorate Types and Development of a Code Frame for the Revised List: A report on the
methodology used to research non-PhD and non-EdD doctorate degrees included in the SED,
along with the decisions made regarding their future inclusion/exclusion (Williams, Welch,
Lisek, and Loew, 2007).
Survey of Earned Doctorates: Qualitative Testing for the SED Questionnaire: Key
findings and recommendations from six Focus Groups and 17 cognitive interviews conducted on
the SED, with Special Emphasis on potential methods to ask for salary. The purpose was to
inform the questionnaire revisions for the SED 2007-2009, including a salary question and a
truncated social security question. (Hess, Hoffer, et al., 2005)

Page 1 of 4

The Salary Panel Meeting for the Survey of Earned Doctorates: A NORC report on the
rationale, limitations, and next steps should NSF decide to add a question on expected salary on
the Survey of Earned Doctorates. Includes recommendations from the 8 panelists from
government, academe, and professional associations. (Hess, Hoffer, 2004)
Survey of Earned Doctorates: Testing a Web Collection Process in Institutions: a report on
the initial beta-test of the process of incorporating a web-based survey for the Survey of Earned
Doctorates and the development of an electronic institutional interface; describes the challenges
of incorporating into the procedures used for the collection of the SED by a variety of Graduate
Schools. (Simko, Hess, 2003)
Evaluation of the SED Educational History Question: an analysis of the effect of question
changes regarding the capture of educational history information in the Survey of Earned
Doctorates. (Hoffer, Hess, Sederstrom, Selfa, Welch, Bullock, 2002)
Survey of Earned Doctorates: Cognitive Interviews on the SED Web Questionnaire:
discussion of results of cognitive interviews on the 2002 SED Web questionnaire. (Simko, Hess,
Ahsan, and Hoffer, 2002)
Results from Cognitive Interviews of NSF Earned Doctorates Web Survey: discussion of
results of cognitive interviews on the 2002 SED web and paper versions. (Althheimer and
Dillman, 2002)
Working Focus Group Report of Institutional Contacts for the Survey of Earned
Doctorates: results of a technical experts group meeting. (Friedman, Hess, and Hoffer, 2002)
Summary of Respondent Comments: Survey of Earned Doctorates, 1999- 2001: qualitative
analysis of respondent verbatim comments spanning 1999 - 2001 survey forms. (Alfred-Liro,
Hill, Reyes, and Hess, 2003)
Analysis of the Potential Undercoverage in the SDR from Double Doctorates: a report
showing that .05 percent of doctorates eligible for the SDR sample are excluded from the SED
universe because they earned a S&E doctorate after earning a non-S&E doctorate, which does
not create an undercoverage problem for the SDR. (Harris/Bouzouth/Hill, 2002)
Analysis of Data from the SED and the SDR Concerning Disabilities: a report on the
comparison of somewhat different methods of measuring disability in the SED and the SDR
showing that disability on-set is primarily with age in the SDR. (Hill/Green, 2001)
SED 2000 Web Survey Instrument Development and Beta Test: Final Results Report:
report on initial development of setting up a SED Web questionnaire, devising a system of access
to the instrument, capturing the data from a small group of respondents, and folding in these data
to the annual SED data base. (Hoffer, Nichols, et al., 2001)

Page 2 of 4

Development/Testing of New Locating Method for Survey of Earned Doctorates
Nonrespondents Using Dissertation Abstracts-On Line: measurement of the efficiency of
using the Dissertation abstracts electronic system for locating new doctorates who had not
received a Survey of Earned Doctorates form from their university (Green, 2001)
Summary of Respondent Comments: Survey of Earned Doctorates, 1999: qualitative
analysis of respondent verbatim comments written on 1999 survey forms. (Alfred-Liro and Hill,
2001)
Citizenship and Race/Ethnicity of Ph.D. Degree Recipients: A Comparison of Data from
the University of California and Survey of Earned Doctorates: discussion of results of a
validation study of race/ethnicity and citizenship data, comparing University of California
administrative records and Survey of Earned Doctorates data. (Sui, et al., 2000)
Report on Cognitive Research for the 2000 SED Questionnaire Development Task:
discussion of results of cognitive interviews on proposed SED 2001 item revisions and additions
with 13 doctoral candidates. (Dugoni, Lee and Baldwin, NORC, 1999)
A Qualitative Pre-Test of the Revised Survey of Earned Doctorates, Using Three Focus
Groups: presented the results of three focus groups testing the revised Survey of Earned
Doctorates form. Results were cumulative because modifications were made to the questionnaire
after each focus group was held. (Policy Research Methods Inc., 1996)
Advisory Committee on the Content of the Survey of Earned Doctorates: recommendations
of a large panel representing users of the Survey of Earned Doctorates concerning the content of
the survey (NRC, 1996)
Technical Panel Review of the Content of the Survey of Earned Doctorates: outside panel
recommendations for changes to the survey form and content. (NRC, 1993 and 1995)
Analysis of the Stay Rates of Foreign Doctorate Recipients from U.S. Universities: data
from the Survey of Earned Doctorates were matched with Social Security data to provide new
information on this topic as well as information on the accuracy of the postgraduation plans
section and the Social Security Numbers provided by foreign citizens when they completed the
Survey of Earned Doctorates. (Finn, ORISE, 1995 and 1997)
Review of the Format of the Survey of Earned Doctorates: Professor Don Dillman made
recommendations for changes to the questionnaire format at the request of the NSF (Dillman,
1995)
Validation Study of the Survey of Earned Doctorates: a major report attempting to validate
most sections of the survey, but focusing on assessing the validity of questions on Sources of
Support and Postgraduation Plans. Also described the results of “think-aloud” interviews
conducted with a sample of doctorate recipients. (NRC, 1994)

Page 3 of 4

Investigation of the Computerized Administration of the Survey of Earned Doctorates:
examined the possible impact on the graduate school and students of having an electronicallyadministered Survey of Earned Doctorates. (Sun-Guen Baek, Graduate Division, U. of
California at Berkeley, 1994)
Federal Agency Sponsors Focus Group on the Content and Use of the SED: described the
focus group meeting of the Federal agency sponsors at Airle House to discuss the issues, content,
and uses of the questions in the Survey of Earned Doctorates.
(NRC, 1994)
Validation of the Sources of Support Question on the Survey of Earned Doctorates by
Comparison with University Records: a detailed validation of a problem question on the SED,
the sources of financial support, via a comparison with the individual records of graduates of the
University of California at Berkeley. (Nerad, U. of California at Berkeley, 1993)
Comparison of the Data on Mathematics Doctorates Collected by the Survey of Earned
Doctorates and the American Mathematical Society: detailed comparison of the results of
two collections of data on mathematics doctorate awards by institution. Conducted by IPA to
NSF. (Maxwell, Hill, and Thurgood, 1993)
Evaluation Reports, Quality Profiles for the Survey of Earned Doctorates: description of the
methodology, coverage, survey response, and item response for each year of the Survey of
Earned Doctorates from 1991 to 1996. (NRC), 1997 to 2001 (NORC)
Coder Perceptions of the Survey of Earned Doctorates Questionnaire: summarized the
perceptions of the coders of the over 40,000 questionnaires for the Survey of Earned Doctorates
for indications of respondent confusion, error patterns, and the effects of formatting. (NRC,
1992) (NRC, 1996)

Page 4 of 4