Early Career Doctorates Project (ECD)

Attachment B – Protocol for Cognitive Interviews

NSF Develop Strategies for Contacting Early Career Doctorates

Introduction

The National Science Foundation (NSF) is preparing for a national survey of postdocs, faculty and nonfaculty researchers, the Early Career Doctorates Study (ECDS). The purpose of the ECDS will be to learn more about the career paths and research opportunities for recent doctorate recipients. During this planning phase, NSF would like to learn about the best ways to contact early career doctorates, inform them of the study, and encourage participation. RTI International is conducting these interviews for NSF as part of this effort. I will be asking you a series of questions related to contacting strategies for the ECDS. There are no wrong answers.

[Informed consent form – Attachment B.1]

Do you have any questions before we start?

[Begin audio recording]

First I would like to get some basic background information about your current position and your education. We will use this information to help with our analysis of the results of these interviews.

[Complete participant's background information form – Attachment B.2]

Factors that Influence Participation

Now I have some questions about factors that may influence your decision to participate in the Early Career Doctorate Study if you were selected to do so.

1. What information would you need to help you decide whether to participate or not?

2. Forgetting for a moment how NSF defines “early career doctorate,” what does the phrase “early career doctorate” mean to you? How would you define “early career”? How would you define “doctorate”? Do you consider yourself an early career doctorate? Why or why not?

3. Would the fact that the ECDS is being conducted for the National Science Foundation influence the likelihood of your participation? The National Institutes of Health? How and why?

4. Have you ever been asked to participate in any other NSF survey such as the Survey of Earned Doctorates (SED) or the Survey of Doctorate Recipients (SDR)? If so, did you participate? Why or why not?

5. Have you ever been asked to participate in any other surveys? Have you completed any of these? If not, why not? If so, why did you choose to complete the survey(s) that you did? How did you complete the survey (web, paper, telephone)?

6. Would having $10 donated to a charity of your choice make you more likely to respond? How about receiving a $10 gift card? If you had a choice, which would you choose and why?

7. Are there certain times of the year that would be especially burdensome for you to complete a 30-minute survey? If so, what are they and why?

8. On a weekday during the summer, what is your typical schedule and what are your typical activities?

9. Where do you typically work during the summer? From campus (office, lab, or classroom)? From home? From another location? Where are you in the Fall? Winter?

ECDs use of and management of mail and email

Next I would like to ask you a few questions about your use of campus mail and email, two ways in which we have thought about contacting early career doctorates.

First, I’ll ask you about campus mail.

1. Do you access your campus mailbox differently in the summer compared to fall and spring semesters (e.g., how often check, and at what time)?

Now a question about your use of email.

2. Do you access email differently in the summer compared to fall and spring semesters (e.g., how often check, and at what times)?

Influence of Institutional Appeal

NSF hopes to get endorsement for the ECDS from a high ranking university official at each academic institution. So I would like to find out which of these officials have name recognition among postdocs, faculty and nonfaculty researchers.

[Show list of officials for their university and ask them to circle the names they recognize – Attachments B.3.1 and B.3.2]

Now let’s focus only on the people whose names you circled.

1. Would the endorsement of the study by any of these people make you more likely to participate in the study? If not, why not?

2. If so, how would you order these people in terms of their influence on your participation (1=most influence)? Why did your order them in this way?

3. How often do you receive an email from the person you ranked first?

4. Do you read these emails? If yes, why? If not, why not?

5. How often do you receive a letter in campus mail from the person you ranked first?

6. Do you read these letters? If yes, why? If not, why not?

7. Is there someone else at your institution whose endorsement of the study would make you more likely to participate? If so, who and why?

8. Do you think an endorsement from someone at your institution would influence your answers if you did participate? If so, how?

Preferred mode of administration

Earlier you told me the kind of information you would need to know to help you decide whether to complete a survey. Now, for the next questions let’s assume we are talking about a 30-minute survey that you are interested in completing.

1. If you were given the option of completing this 30-minute survey in each of the following ways, which would you be most likely to choose? Least likely? Why?

Web survey

Paper and pencil questionnaire

One-on-one telephone interview

2. Would you not complete a survey if it was only offered in one of these particular modes? If so, which one(s) and why?

Email subject lines

NSF and RTI are considering variations on the subject line for the email invitation. Please rank the following email subject lines/senders in order of your likelihood of opening the email where 1 = most likely. When you are done I will ask you a few follow-up questions.

[Show list of subject lines/senders – Attachments B.4.1 and B.4.2]

1. Why did you order them in this way?

2. What factors would make you more likely to open the message?

3. What factors would make you less likely to open the message?

4. Are there any that you wouldn’t open? If so, why?

Entry to web survey

Now I would like to talk about the initial screen that ECDs will see when they begin the web survey.

[Show screen shot printed on paper]

1. What information would you want on this screen?

2. Is there any information that you would like to see here that is not? If so, what?

3. Is there any information here that you do not think is needed or would be better in either the invitation email or on the screen where you requested your credentials? If so, what? Would you like this information somewhere else? If so, where?

4. Would you have preferred to have been taken directly to the first question? Why or why not?

4