Methodological Research Concerning the SED

Methodological Research Concerning the Survey of Earned Doctorates

Note: Reports available upon request from Mark Fiegener (mfiegene@nsf.gov).

Missing Information Letter Experiment Memo: A memo with results of the two part experiment conducted on missing critical item follow-up for web respondents during AY 2012. (Welch and Schacht, 2012)

Analysis of Responses to “Months Covered” by Reported Salary on the SED: The memo compared the responses to the question on “how many months does this salary cover” on the SED to a similar question on SDR and NSOPF controlling for respondent characteristics as much as possible. The report recommended cognitive interviews for further analysis before revising the SED salary question. (Hoffer and Groenhout, 2012)

Cognitive Interview Report on Adopting the Web Instrument: A report with recommendations for future outreach efforts from the interviews conducted with three groups of institutions (1) institutions that adopted the web instrument from NORC’s outreach efforts (2) institutions that didn’t adopt the web and (3) institutions that had not yet been part of the web outreach efforts. (Bartolone, 2012)

Defining and Measuring Interdisciplinary Research: A Review of the Literature and Issues for Consideration: The report reviews the literature on defining and measuring interdisciplinary research, identifies obstacles to the definition and measurement of the concept, and presents issues to consider in future efforts to develop better definitions and measurements. (Millar, 2012)

In-depth Analysis of the SED instruments (paper, web, CATI): A comparison of the three modes of data collection for SED along with a comparison of the SED to the SDR web instrument to determine differences that could impact data quality and outline best practices in web design. An additional report was also prepared on possible experiments. (Olson, Smyth, and Kasabian, University of Nebraska – Lincoln, 2011)

Impact of the 2010 Ineligible EdD Degrees on the DRF Memo: An analysis of the potential impact to the 2010 data because of the decision to remove non-research doctor of education degrees from the SED. (Groenhout and Sederstrom, 2011)

Survey of Earned Doctorates 2009 Imputation Experiment: Methodology, tables and findings from the imputation experiment with the 2009 SED final data. (Welch, 2011)

Disclosure Avoidance for Salary Medians in SED Reporting Memo: An analysis of the disclosure avoidance techniques for salary medians under alternative median calculation methods. (Hedberg, Groenhout, 2011)

Analysis of Interdisciplinary Doctoral Research Data from the Survey of Earned Doctorates: An analysis of the use of the “field of dissertation research” items for assessing the nature and prevalence of interdisciplinary research conducted by doctorate recipients, including the results of cognitive interviews with 30 recent doctorate recipients. (Millar and Dillman, 2010).

Survey of Earned Doctorates Scanning Experiment Phase 2 – Results: Findings and recommendations from the optical scanning experiment conducted during AY2009. (Webber, Williams, Murphy, and Ahsan, 2010).

Old Case CATI Feasibility Test Report: A report on the methods employed during the OCC feasibility test and the outcomes of the test, including recommendations for future work. (Dominguez, Hoffer, Jimenez, Latter, 2009).

Are Individuals Able to Accurately Report Whether Their Work Has Been Supported by the 2009 American Recovery and Reinvestment Act? Reports findings from a survey of households in Washington state – and from cognitive interviews of faculty, post-doctorates, and research assistants employed at Washington State University – that were conducted to learn about the validity of results obtained from asking ARRA survey questions. (Millar, Messer, Genter, and Dillman, 2009)

Survey of Earned Doctorates Doctorate Degree Types: A review of the Doctor of Education (Ed.D.): Findings and recommendations from a review of each Ed.D. program in the SED universe to evaluate whether it offers a research or professional doctorate. (Groenhout, 2009).

MIT 2008-2009 Item Response Rate Analysis Memo: Findings from a comparison analysis of the 2009 MIT hybrid data collection instrument and the 2008-09 MIT data collected by SED data collection instruments. (Latter and Groenhout, 2009)

Results From the Survey of Earned Doctorates Web Survey of SED Data Users: Key findings from a web survey of SED data users on their reactions to small cell data suppression. (Simko, Hess, and Dominguez, 2008).

Results From the Survey of Earned Doctorates Cognitive Interviews: Key findings and recommendations from a set of 20 cognitive interviews conducted on the SED with emphasis on the reaction to new formatting. (Simko, Hess, and Dominguez, 2008).

Survey of Earned Doctorates: Results from the Survey of Earned Doctorates Scanning Experiment: Findings and recommendations from the optical scanning experiment conducted during AY 2007. (Webber, Lashley, and Ahsan, 2008).

Survey of Earned Doctorates: Report on the Impact of the Introduction of a four-digit SSN item on the SED: A report on the methodology and analysis of the four-digit SSN experiment conducted in AY 2006. (Welch, Groenhout, Webber, and Williams, 2008).

Summary of Respondent Comments on the Survey of Earned Doctorates: 2001- 2007: qualitative analysis of respondent verbatim comments spanning 2001-2007 survey forms. (Lashley, Hess, and Reyes, 2008).

Survey of Earned Doctorates: Evaluation of the Addition of Questions on Expected Salary to the Survey of Earned Doctorates in 2008: Comprehensive findings and recommendations from the various interviews, focus groups, validations tests, and quantitative experiments conducted to assess the addition of a salary question to the 2008 questionnaire. (Latter, Hess, and Webber, 2007).

SED Research Doctorate Types: Procedures Followed in the NORC Review of Research Doctorate Types and Development of a Code Frame for the Revised List: A report on the methodology used to research non-PhD and non-EdD doctorate degrees included in the SED, along with the decisions made regarding their future inclusion/exclusion (Williams, Welch, Lisek, and Loew, 2007).

Survey of Earned Doctorates: Qualitative Testing for the SED Questionnaire: Key findings and recommendations from six Focus Groups and 17 cognitive interviews conducted on the SED, with Special Emphasis on potential methods to ask for salary. The purpose was to inform the questionnaire revisions for the SED 2007-2009, including a salary question and a truncated social security question. (Hess, Hoffer, et al., 2005).

The Salary Panel Meeting for the Survey of Earned Doctorates: A NORC report on the rationale, limitations, and next steps should NSF decide to add a question on expected salary on the Survey of Earned Doctorates. Includes recommendations from the 8 panelists from government, academe, and professional associations. (Hess and Hoffer, 2004).

Survey of Earned Doctorates: Testing a Web Collection Process in Institutions: a report on the initial beta-test of the process of incorporating a web-based survey for the Survey of Earned Doctorates and the development of an electronic institutional interface; describes the challenges of incorporating into the procedures used for the collection of the SED by a variety of Graduate Schools. (Simko and Hess, 2003).

Evaluation of the SED Educational History Question: an analysis of the effect of question changes regarding the capture of educational history information in the Survey of Earned Doctorates. (Hoffer, Hess, Sederstrom, Selfa, Welch, and Bullock, 2002).

Survey of Earned Doctorates: Cognitive Interviews on the SED Web Questionnaire: discussion of results of cognitive interviews on the 2002 SED Web questionnaire. (Simko, Hess, Ahsan, and Hoffer, 2002).

Results from Cognitive Interviews of NSF Earned Doctorates Web Survey: discussion of results of cognitive interviews on the 2002 SED web and paper versions. (Althheimer and Dillman, 2002; draft version).

Working Focus Group Report of Institutional Contacts for the Survey of Earned Doctorates: results of a technical experts group meeting. (Friedman, Hess, and Hoffer, 2002).

Summary of Respondent Comments: Survey of Earned Doctorates, 1999- 2001: qualitative analysis of respondent verbatim comments spanning 1999 - 2001 survey forms. (Alfred-Liro, Hill, Reyes, and Hess, 2003).

Analysis of the Potential Undercoverage in the SDR from Double Doctorates: a report showing that .05 percent of doctorates eligible for the SDR sample are excluded from the SED universe because they earned a S&E doctorate after earning a non-S&E doctorate, which does not create an undercoverage problem for the SDR. (Harris/Bouzouth/Hill, 2002).

Analysis of Data from the SED and the SDR Concerning Disabilities: a report on the comparison of somewhat different methods of measuring disability in the SED and the SDR showing that disability on-set is primarily with age in the SDR. (Hill/Green, 2001).

Development/Testing of New Locating Method for Survey of Earned Doctorates Nonrespondents Using Dissertation Abstracts-On Line: measurement of the efficiency of using the Dissertation abstracts electronic system for locating new doctorates who had not received a Survey of Earned Doctorates form from their university (Green, 2001).

Summary of Respondent Comments: Survey of Earned Doctorates, 1999: qualitative analysis of respondent verbatim comments written on 1999 survey forms. (Alfred-Liro and Hill, 2001).

Citizenship and Race/Ethnicity of Ph.D. Degree Recipients: A Comparison of Data from the University of California and Survey of Earned Doctorates: discussion of results of a validation study of race/ethnicity and citizenship data, comparing University of California administrative records and Survey of Earned Doctorates data. (Sui, et al., 2000).

Report on Cognitive Research for the 2000 SED Questionnaire Development Task: discussion of results of cognitive interviews on proposed SED 2001 item revisions and additions with 13 doctoral candidates. (Dugoni, Lee, and Baldwin, NORC, 1999).

Advisory Committee on the Content of the Survey of Earned Doctorates: recommendations of a large panel representing users of the Survey of Earned Doctorates concerning the content of the survey (NRC, 1996).

Analysis of the Stay Rates of Foreign Doctorate Recipients from U.S. Universities: data from the Survey of Earned Doctorates were matched with Social Security data to provide new information on this topic as well as information on the accuracy of the postgraduation plans section and the Social Security Numbers provided by foreign citizens when they completed the Survey of Earned Doctorates. (Finn, ORISE, 1995 and 1997).

Evaluation Reports, Quality Profiles for the Survey of Earned Doctorates: description of the methodology, coverage, survey response, and item response for each year of the Survey of Earned Doctorates from 1991-1992 (NRC), 1997 to 2001 (NORC).