Memorandum

MEMORANDUM

Date: July 16, 2008

To: Shelly Wilkie Martinez, Desk Officer

Office of Management and Budget

From: Lynda T. Carlson, Director

Science Resources Statistics Division

Via: Suzanne Plimpton, Clearance Officer

National Science Foundation (NSF)

Subject: Notification of data collection under generic clearance

The purpose of this memorandum is to inform you of NSF’s plan to conduct cognitive research for the NSF Survey of Earned Doctorates (SED) under generic clearance for survey improvement projects (OMB #3145-0174). This cognitive research will be very useful for studying and revising the latest SED questionnaire to improve existing survey items. The results will inform the design of the 2010 SED questionnaire that will be submitted for OMB clearance in December 2008.

The results of this research are intended for internal SRS use and they will not be used to produce any official releases.

Background

The Survey of Earned Doctorates has been conducted annually since 1957. The survey collects data on the number and characteristics of all individuals receiving research doctoral degrees from all accredited U.S. institutions. The results of this annual survey are used to assess characteristics and trends in doctorate education and degrees. The current SED survey questionnaire is provided for reference in Attachment B.

In preparation of the 2010 SED questionnaire, NSF wants to conduct a series of cognitive interviews in August 2008 to provide information for the 2010 survey design. The schedule for this task will be set so that all recommendations can be implemented in accordance with the OMB submission deadlines.

Research Plan

The goal is to complete a total of 20 cognitive interviews with recent doctoral graduates, all of whom will have previously completed the 2008 SED questionnaire. The approach will incorporate the entire SED survey questionnaire, targeting certain pre-determined items. Issues of particular interest to the cognitive research include:

Item/Area for Cognitive Testing	Justification
Reactions to changes in the 2010 questionnaire, which include layout changes (e.g. eliminating the ‘flip out’ page with the taxonomy of the fields of study, changing the location of the NSF/SRS letter to the graduates). SRS also wants to test the reactions to the scannable format of the questionnaire, in particular the presence of bar codes. A mock-up version of the 2010 version of the form will be used.	The 2010 version of the survey is currently in the final redesign phase and it is important to test respondents’ reactions to the questionnaire changes.
Review of current respondent consent language on the front cover of the survey	SRS wants to investigate how respondents perceive the pledge of confidentiality in the front of the questionnaire, particularly looking at whether the respondents focus on the pledge when they start responding, whether they go back to it later on or not, and also explore the reactions and feelings about the current pledge when they do read it.
Interpretation of the term “interdisciplinary” in question A2	Interdisciplinary degrees have become more prominent over the last few years and SRS wants to explore respondents’ understanding of the term when completing the survey.
Understanding of the term “tuition remission” in question A4	This question was slightly redesigned in recent years by adding the word “waiver” in parenthesis. The purpose was to further clarify the word “remission.” The goal of including this term for cognitive testing is to be sure that the respondents are considering the acceptable types of remission, meaning they understand what is and what is not considered “tuition remission.”
Wording and response categories related to the post-graduation salary items in questions B8 and B9	The salary questions were cognitively tested in 2005, prior to their addition to the survey. While the testing of this item resulted in useful information and confirmed respondents’ interpretation of the language, it will be helpful to follow up on its use within the survey and affirm its functionality and design.
Wording for the time-to-degree answers in question A8, as well as responses to questions A11 through A13.	The time-to-degree questions A8 and A11 through A13 are multi-item in their design. Because of the importance of the combination of these questions to calculate time-to-degree, it will be helpful to confirm the layout, design, and interpretation of these items.

The SED contractor, NORC at the University of Chicago, will conduct the cognitive interviews in their offices in Chicago. NORC will draw a sample of potential interview participants from respondents to the 2008 SED who graduated from Chicago area universities, and/or indicated on their SED form that they planned to reside in the Chicago area post-graduation. Also, because one of the issues of interest focuses on the interpretation of the term “interdisciplinary” in question A2, NORC will try to oversample respondents who indicated their dissertation research was interdisciplinary.

Once the sample is drawn, NORC will begin the recruitment process. Information collected from the completed SED survey will be used to locate and recruit individuals on the list. Initial contact efforts will be made via email; if unsuccessful, attempts to reach them via mail and telephone contact information will follow. Contacted individuals expressing interest in participating in the cognitive interviews will be asked to confirm background information they provided in their SED form and to indicate their interview time preferences. The background information will also assist NORC in selecting a diverse group of interviewees for the cognitive interviews.

Data collection activities are expected to begin in August 2008, with interviews occurring over the following 4 weeks. Interviews will be scheduled during the day or in the evening, based on respondent convenience and will last no longer than 90 minutes each. All interviews will be videotaped for documentation purposes and to assist in accuracy and note-taking. Once all interviews and documentation are complete, NORC will prepare a final report.

Data Collection Instrument

A draft of the proposed protocol and NORC’s statement of informed consent are provided in Attachment A. The participant background sheet used to confirm the background information of the prospective interviewees is also provided in Attachment A.

Burden Information

NORC plans to conduct 20 cognitive interviews. The estimated time for completion of an interview is 90 minutes, including approximately five minutes confirming the interviewee’s background information prior to the interview. Therefore, we estimate a total burden of 30 hours.

Incentive Payments

Respondents who are selected as participants will be offered a $75 incentive for their participation in the interviews. This is the same incentive as was offered in the last round of cognitive interviews concerning the salary question, held during June, 2005. Offering this incentive will enable NORC to recruit participants from a broad spectrum of fields of study, including Science and Engineering (S&E) doctorate graduates whose salaries are higher.

Contact Information

The contact persons for questions regarding this data collection are:

Mark Fiegener or Nancy L. Leach

Project Manager Program Director

Survey of Earned Doctorates Human Resources Statistics Program

Science Resources Statistics Division Science Resources Statistics Division

National Science Foundation National Science Foundation

703-292-4622 703-292-7768

mfiegene@nsf.gov nleach@nsf.gov

Attachments (2)

cc: Fran Featherston