November 6 Memo, revised December 16

MEMORANDUM

Date: December 15, 2009 (revised)

To: Shelly Martinez, Desk Officer

Office of Management and Budget

From: Lynda T. Carlson, Division Director

Division of Science Resources Statistics

Via: Suzanne Plimpton, Reports Clearance Officer

National Science Foundation

Subject: Notification of information collection under generic clearance

The purpose of this memorandum is to inform you of NSF’s plans to conduct a series of cognitive interviews to develop new questions related to the 2009 American Recovery and Reinvestment Act (ARRA, i.e. Stimulus Bill) under the generic clearance for survey improvement projects (OMB number 3145-0174). The proposed new questions would be added to the Survey of Doctorate Recipients (SDR), National Survey of College Graduates (NSCG), and the National Survey of Recent College Graduates (NSRCG), collectively known as the Scientists and Engineers Statistical Data System (SESTAT) surveys.

Background

The Division of Science Resources Statistics (SRS) uses the three SESTAT surveys to produce estimates on different portions of the U.S. science and engineering workforce. The SESTAT surveys are generally conducted every two years. The SESTAT surveys contain two types of questions: core and module. Core questions are those that are asked in every survey round to understand the basic employment, educational and demographic characteristics of the science and engineering labor force. Module questions are periodically asked on the surveys to address special topics. For the 2010 SESTAT surveys, SRS would like to include a module on the ways in which ARRA has affected U.S. scientists and engineers. Obtaining reliable estimates from ARRA-related questions may be particularly problematic given that people may or may not know whether they have received ARRA funding. To begin to understand and to determine how to collect valid and reliable data, SRS proposes using cognitive interviews. For these cognitive interviews SRS will use administrative records to identify participants known to have received or to have not received ARRA funds. Selecting participants using administrative records will provide SRS with insights about the potential bias in responses, and some of the reasons. In addition, before implementing an ARRA-related SESTAT module SRS will further explore methods of data validation to ensure that the ARRA-related questions provide reliable measures.

The SESTAT surveys are next scheduled to be administered in October of 2010, using a multi-mode approach of a self-administered mail survey, computer-assisted telephone interviews and a web survey. The 2010 SESTAT survey samples will consist of individuals who have at least a bachelor’s degree in science, engineering and health (SEH) fields and/or work in these fields.

Proposed Questionnaire Testing

SRS plans to conduct cognitive interviews to evaluate the proposed questions to determine if they can adequately capture the effect of ARRA on survey respondents. Through cognitive interviews, SRS will test the performance of the proposed ARRA questions to ensure that the questions perform as intended and are clearly understood by the respondents. Administrative records will be used to recruit participants known to have received or to have not received ARRA funds which will allow validation for some of the participants’ responses, SRS also hopes to identify and document the nature and extent of any problems with the questions. This information can be collected through observation of respondents during the cognitive interviews, and probing with follow-up questions. NSF plans to use the results of the cognitive interviews to further refine the ARRA questions, including response category wording.

Recruitment will occur throughout the Washington State University Pullman-Moscow area for people with a college degree who are working in science or engineering related fields. At least half of the participants will have received Federal stimulus funding through either grants or contracts according to the institution’s administrative records.

Potential participants will be called and informed about the goal of the study, which is to evaluate a set of proposed questions for the SESTAT surveys. Explanations will include the voluntary nature of participation in the cognitive interview, the nature and length of the interview, and the providing of $35 in appreciation for the help they provide. They will also be asked to sign a consent form prior to the interview, a draft of which is attached.

Burden Hours for Cognitive Interviews

A maximum of 20 cognitive interviews are planned. The estimated completion time of each interview is approximately 60 minutes. The total maximum estimated burden for this research is 20 hours. Each interviewee will receive $35 for their participation.

Clearance Package

Included in this clearance package is the draft cognitive interview questionnaire. If any significant changes are made to this instrument prior to testing, SRS will submit a revised version to OMB prior to conducting the cognitive interviews.

The contact person for questions regarding this research is:

Emilda Rivers

Program Director

Human Resources Statistics Program

Division of Science Resources Statistics

National Science Foundation

4201 Wilson Boulevard, Suite 965

Arlington, VA 22230

703-292-7773

erivers@nsf.gov

ATTACHMENT

CC: Emilda Rivers

Mary Frase

Dan Foley

Kelly Kang

Steven Proudfoot

Fran Featherston