MEMORANDUM
August 2, 2013
To: Shelly Martinez, Desk Officer
Office of Management and Budget
From: John R. Gawalt, Director
National Center for Science and Engineering Statistics
Via: Suzanne Plimpton, Clearance Officer
National Science Foundation
Subject: Notification of information collection under generic clearance
The purpose of this memorandum is to inform you that the National Center for Science and Engineering Statistics (NCSES) of the National Science Foundation (NSF) plans to conduct cognitive interviews for the Early Career Doctorates Project (ECDP) under the generic clearance for survey improvement projects (OMB number 3145-0174).
The objectives of the cognitive interviews are to evaluate survey questions for correct interpretation and to assess the ability of early career doctorates (ECDs) to accurately answer the survey questions. For these cognitive interviews, NSF plans to obtain feedback on the questionnaire developed for the ECDP and revised based on the results of a prior methodological study. The results of the cognitive interviews will be used to refine the questionnaire for an upcoming ECDP pilot and future full-scale data collection effort.
NCSES is gathering information about ECDs including postdoctoral researchers (postdocs) and other doctorate-holding nonfaculty researchers who have received doctorates in the last ten years. Most recently, NCSES completed a methodological study (OMB #3145-0174 in a memo dated 05/18/2012) in which a survey was administered to these individuals in three settings: (1) institutions that participate in the National Science Foundation –National Institutes of Health (NIH) Survey of Graduate Students and Postdoctorates in Science and Engineering (GSS), (2) Federally Funded Research and Development Centers (FFRDCs), and (3) NIH intramural research programs. As with previous research, NCSES engaged the community on next steps for the project and questionnaire development given the results of the methodological study.
The ECDP questionnaire that will be used for the cognitive interviews was refined based on results from the methodological study. The results indicated the need to select additional questions from the Survey of Earned Doctorates, the Survey of Doctorate Recipients, and from a questionnaire developed for the Careers of Doctorate Holders (CDH) project, an international collaboration of the Organisation for Economic Co-operation and Development. Using questions from these sources will ensure the ECDP meets a goal of collecting nationally and internationally comparable data on the labor market, career path, and mobility of doctorate holders.
RTI International will conduct this work under contract to NCSES.
Proposed Methodology
There will be up to 32 individual cognitive interviews conducted, each lasting approximately 1½ hours. Interviews will be conducted in two rounds. For each round, the participants will be asked to complete a subset of survey questions from the superset of questions (see attached). After completing a survey question, the participant will be asked follow-up probes to assess his/her interpretation of the question wording and response options and his/her response process. In addition for some survey questions, the participants will be instructed to think aloud as they interpret the question, retrieve the information, form a response, and select a response option. After the first round, survey questions will be revised as needed and retested in the second round. Some survey questions may perform well in the first round and not require further testing.
Participants will be recruited from the GSS-eligible institutions with respect to position title (postdoc, faculty and other doctorate-holding nonfaculty researchers), country of origin of doctorate degree, citizenship and field of degree. Participants will be recruited from several institutions within a half hour drive of RTI International’s campus. Based on prior challenges with recruiting this population, participants will receive a $50 incentive. The $50 incentive will assist in compensating for these 90 minute interviews a diverse set of participants who are highly educated, relatively highly compensated and extremely busy individuals.
The recruitment strategy will include email announcements and flyers to inform candidates about the goals of the project, the purpose of the cognitive interviews, and the incentive to be provided at the completion of the interview. Email announcements will be sent to postdoc associations, research institutes, and faculty listservs with the permission of the listservs’ moderators. In addition, flyers will be posted on accessible bulletin boards in locations likely to be visited by ECDs, such as the postdoc association office, research centers, and international association offices. If enough candidates have not been recruited using these methods, participants will be asked if they know of any other ECDs who may be interested in participating. Potential participants will be asked to complete a short web form to provide basic contact and background information about themselves. Alternatively, prospective participants will be provided the option to reply by email or telephone. The information collected at this stage will be used to determine eligibility and to select a diverse sample of participants meeting the survey or project criteria. The recruiting materials, including the email announcements and flyer, are provided in Attachment A.
Prior to the interview date, participants will receive a confirmation email. At the start of each interview, an informed consent form will be given and participants will be verbally reminded of their rights to decline participation, that they may end the interview at any time, and that they may choose not to answer any question. In addition, they will be given an opportunity to read the informed consent form and sign it if they choose to participate. The consent form (refer to Attachment B.2) will explain the following: the intention to have a note taker present to allow the facilitator to focus entirely on the discussion; the desire to audio record interviews for later review when details not found in the notes are needed; the possibility that NSF may wish to listen to the interviews for firsthand knowledge of participants’ responses; and the results of the interview will be summarized. In the event that representatives of NSF are present for the interview, participants will receive a version of the informed consent that indicates that they may be observing.
The protocol for the cognitive interviews (see Attachment B) is designed to gather information on how selected survey questions perform. The attached contains a superset of survey questions which are eligible for the cognitive interviews based on results from the methodological study or have been proposed for the upcoming ECDP pilot. The results of the cognitive interviews will be used to revise the questionnaire for the upcoming ECDP pilot and/or the future full-scale data collection effort.
Informed Consent
Before the interview begins, all participants will be provided with an informed consent document explaining their rights, the purpose of the study, the risks and benefits of participating, how their information will be protected, and whom they may call if there are any questions (attachment B-2).
We estimate the total recruitment burden to be 500 minutes or approximately 8 ½ hours (100 prospective participants with each projected to take about 5 minutes to respond to the recruitment announcement).
There will be up to 32 individual cognitive interviews, each projected to be 1 ½ hours long. Thus, the maximum total burden for the proposed cognitive interviews is 48 hours (32 participants x 1.5 hours per interview). There will be a maximum of 32 cognitive interviews, including those who choose to terminate the interviews early.
The total burden (recruitment and cognitive interviews) is estimated to be 56 ½ hours.
To attract a diverse set of participants from this population of busy, educated, relatively highly compensated individuals, cognitive interview participants will receive $50 as an expression of appreciation for participating in the 90 minute interview.
The contact person for questions regarding this research is:
Kelly S. Phou
National Center for Science and Engineering Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
703-292-4722
Attachments
cc: Jeri Mulrow
Stephen Cohen
Emilda Rivers
Rebecca Morrison
Kelly Phou
| File Type | application/msword |
| File Title | OMB Clearance request pdp focus group |
| Author | credline |
| Last Modified By | martinez_r |
| File Modified | 2013-09-20 |
| File Created | 2013-09-20 |