Justification Memo

MEMORANDUM

Date: February 14, 2012

To: Shelly Wilkie Martinez, Desk Officer

Office of Management and Budget

From: John Gawalt, Acting Division Director

National Center for Science and Engineering Statistics (NCSES)

Via: Suzanne Plimpton, Clearance Officer

National Science Foundation (NSF)

Subject: Notification of data collection under generic clearance

The purpose of this memorandum is to inform you of NCSES’s plan to conduct cognitive research for the Survey of Earned Doctorates (SED) under generic clearance for survey improvement projects (OMB #3145-0174). The purpose of this cognitive research is to identify the barriers institutions face in adopting the web survey mode for the SED. The proposed research entails semi-structured interviews with institutional contacts to explore their thoughts on the content of outreach materials and their questions about transitioning to the web SED survey instrument. The results of the research will be made available for improving the outreach materials and protocols that are currently used to encourage institutions to adopt the web survey mode.

The results of this research are intended for internal NCSES use and they will not be used to produce any official releases.

Background

The SED has been conducted annually since 1957. The survey collects data on the number and characteristics of all individuals receiving research doctoral degrees from accredited U.S. institutions. The results of this annual survey are used to assess characteristics and trends in doctorate education and degrees.

Historically, the SED has been conducted with the help of institutional contacts (typically, Graduate School Deans and members of their staff) at the academic institutions who distribute a paper version of the survey to graduate students near the time they are awarded the research doctorate degree. Starting in 2000, the SED began offering a web version of the survey. Initially, the web survey mode was used to assist institutional coordinators in reaching graduates that did not respond to the paper version of the survey. However, owing to technological advances the web survey mode has become an increasingly effective way to reach respondents and administer the SED.

While the web survey mode for the SED does require some additional effort to reconcile completed surveys to the final graduation lists, it has many advantages over the paper survey mode, including:

• The elimination of labor associated with pre-key editing, data entry, and document management;

• A significant decrease in printing and postage costs;

• Improved data quality through controlled skip patterns and response selection;

• Faster access by a requesting institution of raw data on that institution’s research doctorate recipients;

• Improved data security with secure electronic transfer of SED data directly from the respondent to the survey contractor; and

• For the institutions, a reduced survey administration burden.

Furthermore, many institutions have implemented electronic processes for accepting dissertations and other degree completion materials from their graduating doctoral students. These institutions are excellent candidates for adopting the web survey mode for the SED.

Over the past two years the current survey contractor for the SED, the National Opinion Research Center (NORC) at the University of Chicago, has pursued a targeted outreach program to inform 33 of the largest “non-web” institutions of the benefits of the web survey mode for the SED. As of February 2012, 18 of these targeted institutions had either adopted the web survey mode or expressed an intention to do so soon. However, several of the large institutions contacted by NORC in the outreach efforts decided not to convert to the web survey mode, and some of the targeted institutions did not respond to the outreach at all. Given the many benefits associated with the web survey mode, the response to the outreach seems surprisingly low.

Research Plan

Overall, nearly 70 percent of programs participating in the SED, representing over 40% of doctorates recipients, indicate that they still plan to distribute paper survey forms. Before conducting additional outreach to encourage these remaining programs to adopt the web survey mode for the SED, NCSES proposes to conduct research to identify the “barriers to adoption” within institutions. The results of this research would then be applied to improve the materials and protocols used in subsequent outreach efforts.

The proposed research entails conducting semi-structured interviews with institutional contacts from 12 to 15 institutions (one contact per institution). Three types of institutions will be represented:

1. Nonrespondents – four to five institutions that did not respond to previous outreach efforts; the institutional contact (Graduate School staff member) may help articulate the barriers that prevented the institution from adopting the web survey mode for the SED.

2. Positive Respondents – four to five institutions that responded to previous outreach and that either recently transitioned to the web survey mode for the SED or indicated they are likely to transition soon; the institutional contact (Graduate School staff member) may help articulate the factors that triggered the institution’s adoption of the web survey mode for the SED.

3. No Prior Outreach – four to five institutions that are currently using the paper SED form and that were not contacted in earlier outreach efforts; the institutional contact (Graduate School Dean) may help articulate the best protocols for reaching decision-makers and influencing them to transition to the web survey mode for the SED.

The semi-structured interviews (see Appendix for protocol) will explore respondent reactions to the content and design of the outreach materials, thoughts on the appropriate mode of outreach, and questions or concerns with transitioning to the web SED survey instrument.

Data collection activities are expected to begin in March 2012, with interviews occurring over a 3-week period. All interviews will be conducted over the telephone. Each interview will last no longer than 30 minutes, and will be scheduled during the day based on interviewee convenience. Once all interviews are complete, NORC will prepare a memorandum with recommendations for improving the outreach materials and protocol.

Data Collection Instrument

The set of proposed questions for each of the three groups is attached to this memo (see Appendix).

Burden Information

NORC plans to conduct 12-15 semi-structured interviews. The estimated time for completion of an interview is no more than 30 minutes. The time to recruit individuals (via email and telephone) to participate in the study is estimated at 5 minutes for each of 24 candidate participants, for a total estimated recruitment time of two hours. Total burden is estimated at between 8 and 9.5 hours.

Incentive Payments

We do not intend to utilize incentive payments in this research.

Contact Information

The contact persons for questions regarding this data collection are:

Mark Fiegener

Project Officer

Survey of Earned Doctorates

National Center for Science & Engineering Statistics

National Science Foundation

703-292-4622

mfiegene@nsf.gov

Emilda Rivers

Program Director

Human Resources Statistics Program

National Center for Science & Engineering Statistics

National Science Foundation

703-292-7773

erivers@nsf.gov

Attachment