Request for Approval of Methodological Work on the NSF-NIH Survey of Graduate Students and Postdoctorates in Science and Enginee

Memorandum

Date: February 24, 2011

To: Shelly Martinez, Desk Officer

Office of Management and Budget

From: Lynda T. Carlson, Division Director

National Center for Science and Engineering Statistics

National Science Foundation

Via: Suzanne Plimpton, Reports Clearance Officer

National Science Foundation

Subject: Request for Approval of Methodological Work on the NSF-NIH Survey of Graduate Students and Postdoctorates in Science and Engineering (GSS)

The National Science Foundation requests approval of methodological work for the GSS to determine: (1) institutions’ eligibility for the survey and (2) the best procedures to use to efficiently screen potentially eligible institutions. Approximately 500 institutions, primarily master-level institutions, have been identified using Integrated Postsecondary Education Data System (IPEDS) completions data and other sources as offering graduate degrees that may be eligible for the GSS. Eligibility will be determined for the branch campuses and separately organized schools that offer eligible graduate degree programs at the 500 institutions.

Background

The target population for the GSS is defined as U.S. academic institutions that offer graduate degree-credit programs in the sciences and engineering (as defined by NSF) and in health-related fields (as defined by NIH) in the U.S., including post-baccalaureate programs. An institution is considered eligible, or in-scope, if it meets at least one of the following criteria:

• Grants at least one master’s or doctoral degree in at least one program listed in selected NCES Classification of Instructional Programs (CIP) codes.

• Has at least one postdoctoral appointee or non-faculty research staff member conducting research in at least one of the following broad areas: agricultural sciences, computer sciences, engineering, environmental sciences, life sciences (biological and health), mathematical sciences, physical sciences, psychology, and social sciences.

The initial determination of the 500 institutions’ eligibility for the GSS was based on a review of the institutions’ websites for information pertaining to the CIP codes of graduate degree completions reported to IPEDS. While determining the institutions’ eligibility, the survey contractor also identified additional graduate programs that may have been eligible. The review resulted in assigning a code to each institution as “likely to be eligible”, “unlikely to be eligible”, or “undetermined” based on the eligibility of the graduate degree programs listed on the institutions’ website. The results of the initial eligibility determination after the QC review are displayed in Table 1 below.

Table1. Eligibility codes based on reviews of institutions’ websites

Code	Value	Number of institutions
2	Likely to be Eligible	219
3	Unlikely to be Eligible	270
4	Undetermined	41
5	Ineligible (School reported by existing GSS institution)	20
6	Ineligible (School closed)	2
7	Ineligible (No GSS-eligible programs identified at school)	6
	Total	558

We had originally planned to base the decision of whether or not to add institutions to the GSS only on the results of the website reviews. Given degree program variability within and across institutions and their schools, we decided that further screening is necessary to confirm and verify the eligibility of the degree programs offered.

Proposed methodology

Within institutions, schools will be contacted and asked to complete a short screener questionnaire via the web. The initial contact will be with the institutional research office with additional contacts to other offices (such as the graduate school) if the institutional research office contact cannot be identified. Large schools and schools that do not complete the web survey will be asked to complete the screener questionnaire by phone.

For most GSS-eligible fields of study, the screening process will confirm that a graduate degree identified in the IPEDS completion data and the institution website review is offered in that field. However, there are three conditions based on field types which will require additional screening to determine eligibility.

1. Some fields have specific practitioner degrees that are excluded from GSS: architecture, anesthesiology, dental sciences, nursing, ophthalmology, pharmaceutical sciences, veterinary science, clinical medicine (not elsewhere classified) and chemistry.

For these fields, respondents will be asked to list the graduate degree programs offered in the field.

2. In some GSS-eligible fields there are distinctions between research-oriented and practitioner-oriented degrees that are not clear by the name of the degree program. These fields are: nutrition, family and consumer sciences, communication disorders sciences, health-related fields (not elsewhere classified), psychology, political science/public administration.

For these fields, respondents will be asked to answer 4-5 additional items.

3. In two fields, engineering management and management information systems, there is a distinction between whether the degree is primarily a management degree or a science and engineering degree.

For these fields, respondents will be asked if they would categorize the program as a science and engineering program, management program, or multidisciplinary/interdisciplinary program.

The screener questionnaire and eligibility determination criteria are provided in Attachment 1. Answers to the screener questionnaire will be used to determine eligibility based on the criteria provided in Attachment 1. If there is a conflict between the eligibility statuses determined from the screener questionnaire versus the website review, the institution/school will be contacted to obtain additional information related to eligibility criteria. NSF will review the final recommendations made by the survey contractor concerning institutions’ eligibility and will decide which institutions to include in the 2011 GSS.

Results of the screening process will be analyzed to determine the best procedures for screening potentially eligible institutions/schools in future GSS cycles. For example, we will investigate whether there are CIP codes for which the degree program eligibility status was the same across all sources (IPEDS completion, institution website review, and screening survey). This will allow us to streamline the future screening process and increase efficiency.

The tentative schedule for this methodological work is as follows:

Proposed Date	Activity or Deliverable
February 24, 2011	OMB submission for approval
March 10, 2011	OMB clearance
March 22, 2011	Finalize instrument and send letters to presidents of potentially new institutions
March 28, 2011	Send emails to contact persons at potentially new institutions to begin eligibility screening survey data collection
April 11, 2011	Begin phone prompts, answer questions
May 23, 2011	End eligibility screening survey data collection
June 15, 2011	Preliminary institution database and methodology report available to NSF
July 8, 2011	Final database and report available to NSF
August 5, 2011	NSF approval of eligible institutions to be included in 2011 GSS

Response Burden

We estimate approximately 250 burden hours for this methodological work: a 15 minute survey for approximately 1,000 departments in 500 institutions. Approximately 15 minutes per department should provide sufficient time for completing the screening questionnaire. This estimate is covered by the current GSS clearance which includes 360 burden hours for future testing needs.   

Contact Person

Kelly Kang (kkang@nsf.gov, 703-292-7796)

Human Resources Statistics Program

National Center for Science and Engineering Statistics

National Science Foundation

5