Summary of Changes Proposed for 2011 GSS

Attachment 8

Summary of Changes Proposed
for 2011 GSS

Summary of Changes Proposed for 2011 GSS

A. Survey universe change – addition of newly eligible institutions

Add approximately 200 new institutions (approximately 500 units) offering graduate degrees. NSF is currently conducting a GSS eligibility screening survey as part of the GSS Frame Expansion Study (to be completed in August 2011). The institutions determined to be eligible for GSS from the Study will be asked to participate in the 2011 GSS.

B. Instrument changes

No major changes to the items are anticipated. However, some enhancements of the web survey instrument will be made.

B1. Remove remaining ambiguous zeros from the Web survey instrument.

Since the default value in all cells in the question grid is zero, nonresponse is indistinguishable from a response of zero. This is problematic when counts for a subgroup (e.g., men) must be inferred by subtracting counts for the complementary group (e.g., women) from total counts; that is, when counts for one subgroup are not explicitly collected. In these cases, if counts for the complementary group are zero, it is not clear whether all of the total should be applied to the implicit subgroup, or if the total should be distributed across the subgroup and complimentary group through imputation.

The redesign of the Part 2 grids in the 2008 and 2010 survey cycles has eliminated all of this ambiguity except for the count of full-time graduate students by first-time and returning status. During data review and retrieval in the 2009 survey cycle, just over one-quarter of all zero counts for first time full-time students were found to actually be cases of nonresponse that would be imputed.

Therefore, web survey in 2011 GSS will be modified to address this issue by changing the default value in the first-time full-time cells from zero to blank. The GSS web screen shots on this item reflect this change.

B2. Display the highest degree granted on the unit list screen so that the school coordinators may review it when confirming the list.

Currently, the SC only indicates the highest degree granted by a unit when he or she is creating a new unit. Apart from those instances, the SC is unable to view the highest-degree status of a unit in Part 1 section. They may view the highest degree designated for each unit in the Unit Profile in Part 2, but this is not part of the unit list update and is therefore much less likely to be reviewed systematically.

The display of existing unit information was altered to include information on the highest degree offered in the unit, and allow coordinators to edit the information. The Part 1 unit listing in the GSS web screen shots reflects the changes to the display; coordinators can click on the unit to modify information.

B3. Build “degree checks” into the survey instrument for all the fields with excluded degrees.

We have found that some respondents continue to include graduate students pursuing practitioner-oriented degrees despite an effort to provide clear and prominent ‘exclusion’ instructions to the contrary. Therefore, for units in fields with practitioner-based degrees, the respondents will be asked to confirm that the counts reported do not include any graduate students pursuing an excluded degree, and prompt them to remove them in the counts otherwise. Until this is confirmed, the unit would be in error and data submission would not be allowed. An example of this warning is included in the GSS web screen shots.

B4. Tailor the web instrument for schools that have no postdocs or doctorate-holding nonfaculty researchers to reduce burden.

In the current web instrument, schools that have no postdocs or NRFs are required to go thru each postdoc and NFR grids for every unit and indicate so. They are also asked to answer questions about how their school defines a postdoc position, which would not apply to them. This adds unnecessary burden for these schools. To allow SCs to report the absence of postdocs and NFRs more efficiently, the instrument will be tailored: (1) to ask schools that reported no postdocs in the past to confirm that this is still true; (2) for the schools that reported postdocs in the past, to pre-fill the postdoc definition question with their past responses for the respondents to confirm or modify the responses; and (3) to add checkboxes on the unit listing screen to allow SCs to indicate that the unit has no postdocs or NFRs. These changes are reflected in the GSS web screen shots included in the attachments.

C. Changes to Survey Procedures

C1. Create trainings for respondents.

Since the GSS will be adding many newly eligible schools in the 2011 survey cycle, it would be beneficial to develop a training that would acquaint them with the purpose of the GSS, teach them how to navigate the web survey, and provide best practices that will make their work easier and improve the quality of the data collected.

A web-based training will be developed for new respondents that can accessed at any time. A separate presentation will be developed each year for veteran respondents, which highlights the changes from the previous survey cycle. These web-based training modules will be developed during the summer of 2011.

C2. Investigate ways to encourage SCs and PCs to upload data in an effort to reduce respondent burden.

Uploading data from centralized data sources is an efficient method of reporting data for school coordinators at schools with integrated data bases. Smaller schools, non-medical schools, and schools less active in research are more likely to have centralized data bases. We plan to conduct interviews with the SCs at these types of institutions to find out why they do or do not upload data. This research will be conducted using the methodological burden hours requested in this clearance package. With knowledge of the factors that facilitate or deter SCs from taking advantage of this feature, we can design the survey procedures and system to encourage more SCs to upload data.

D. Changes to Definitions

D1. Clarify definitions as recommended in the record-keeping study.

The survey contractor will work with NSF during the spring-summer of 2011 to clarify definitions for some items and will modify the glossary to reflect these changes.

E. Review Taxonomy and CIP-GSS Crosswalk

E1. Review GSS taxonomy in conjunction with frame expansion project.

Any modifications to the GSS taxonomy as a result of the frame expansion project will be incorporated into the 2011 GSS.

E2. Update GSS-CIP crosswalk to 2010 CIP.

NSF is currently developing a crosswalk between GSS field codes and the 2010 CIP codes that can be used by institutions in the 2011 GSS.