Cognitive Research for the NSF Survey of Earned Doctorates

STATEMENT OF INFORMED CONSENT

Survey of Earned Doctorates – Cognitive Interviews

The National Opinion Research Center (NORC) is conducting a series of cognitive interviews for the Survey of Earned Doctorates (SED) on behalf of the National Science Foundation and its sponsors, the National Institutes of Health, the U.S. Department of Education, the National Endowment for the Humanities, the U.S. Department of Agriculture, and the National Aeronautics and Space Administration. The procedures and content of this research have been approved by the U.S. Office of Management and Budget. By participating in this study, you are assisting us in the revision of our primary research instrument for the 2010 academic year. Please read the statements below carefully.

CONFIDENTIALITY

This information is solicited under the authority of the National Science Foundation Act of 1950, as amended. All information collected will be confidential and used only for the purpose of evaluating the SED survey questionnaire. Your identifying information will be removed from any results or reports of data. We want to assure you that anything you tell us or any information that you give us will not be attributed to you individually. The information will only be shared with the research team. We consider the information you choose to share with us personal and confidential. You have the right to refuse to answer any question, and to refuse to participate. However, we expect the outcome of this survey to benefit you in that the results will be used to improve the quality of the data that NSF reports about doctorate recipients.

PROCEDURES

NORC will interview you at its Chicago facilities. You will be asked to complete a videotaped think aloud session, where you will complete the paper-and-pencil questionnaire, and then answer follow up questions about your thought processes and experiences during this exercise. The interview will take no more than 90 minutes of your time.

We would like to audio/video tape this interview. Audio/videotape recording is used to allow the researcher(s) to record and analyze data at a level of accuracy and detail that is otherwise unattainable. Audio/videotape recording the session will allow us to focus on the interview, rather than on taking notes, which enhances the productivity of the session. If at any time during our interview you should feel uncomfortable, you are under no obligation to stay, and are entirely free to leave.

RISKS AND BENEFITS

This survey will take no more than 90 minutes of your time. You are free to refrain from answering any one of the questions, and you may stop the interview at any time. Failure to provide some or all of the requested information will not in any way adversely affect you.

As a token of our appreciation, upon completion of the entire interview, you will be paid $100.00 in the form of a personal check to be mailed to an address which you provide.

_________________________________ ____________________

Interviewer Date

PARTICIPANT STATEMENT

I understand that this session will be videotaped and that it is completely confidential. I have had an opportunity to ask questions about my rights as a participant before I signed this form. I also understand that any further questions I have about this study or about my rights will be answered by the interviewer named above.

_________________________________ ____________________

Participant Date

SED 2008 Cognitive Interview Instructions

During this interview, I am going to ask you to complete a questionnaire and answer some follow up questions. As you complete the questionnaire, I would like you to try to think aloud and verbally share your thoughts and reactions; please try not to censor your thoughts. Please feel free to share all of your opinions and questions about what you are seeing. One of the things I am most interested in is how you are deciding which answer to select. I am going to take notes and will try to respond to your questions when we are finished. When you complete survey, I may ask you to go back to certain items that I would like more information about.

Before we begin, I’d like to go through an exercise to help familiarize you with what I mean by “think aloud”. [PAUSE] Try to visualize the place where you live. Now, think about how many windows there are in that place. [PAUSE] As you count up the windows, tell me what you are seeing and thinking about.

[AFTER PARTICPANT RESPONDS]

That’s an example of what I would like you to try and do as you read through the survey. Do you have any questions before we begin?

[HAND PARTICIPANT THE ENVELOPE]

In this envelope you will find a questionnaire and a pencil. The letter and brochure will provide some background information on the survey. I would like you to take out the questionnaire. Beginning with the first page, please complete the questionnaire using the pencil provided. Please remember to tell me what you are thinking about as you respond to each question. Please feel free to share all comments with me.

[BEGIN COGNITIVE INTERVIEW AND PROBES]

Protocol – Survey Items and Probes

[INTERVIEWER: NOTE THAT PROBES BEGIN ON PAGE 6 OF THIS DOCUMENT].

Cover Page:

1. Name, institution, graduation information

2. Confidentiality/Consent Statement

3. Bar codes/scanning layout

Instructions

Part A – Education

A1. What is the title of your dissertation?

A2. Please write the name of the primary field of your dissertation research.

A3. Please name the department of the university that supervised your doctoral studies.

A4. If you received full or partial tuition remission (waiver) for your doctoral studies, was it…

A5. Which of the following were sources of financial support during graduate school?

A6. Which TWO sources listed at A5 provided the most support?

A7.When you receive your doctoral degree, how much money will you owe that is directly related to you undergraduate and graduate education?

A8.a. Have you received a degree of this type?

A8.b. Month/year degree granted

A8.c. Month/year that you started your degree

A8.d. Primary field of study

A8.e. Field number from list on p.7

A8.f. Institution name

A8.g. Branch or city

A8.h. State or province

A8.i. Country

A9. Excluding those above, have you attained any additional postsecondary degrees?

A10. Was a master’s degree a prerequisite for admission to your doctoral program?

A11. In what month and year did you first enter graduate in any program or capacity?

A12. How many years were you (a) taking courses…, (b) working on your dissertation…?

A13. Was there any time from the year you entered your doctoral program and the award of your doctorate that you were not working on your degree…?

A14. Did you earn college credit from a community or two year college?

A15. Are you earning or have you earned an MD or a DDS?

Part B- Post-graduation Plans

B1.In what country or state do you intend to live after graduation?

B2. Do you intend to take a “postdoc” position?

B3. What is the status of your postgraduate plans?

B4. What best describes your postgraduate plans?

B5. What will be the main source of financial support for your “postdoc”or further training within the next year?

B6. What type of principal employer will you be working for in the next year?

B7. Please name the organization and geographic location where you will work or study.

B8. What will be your basic annual salary for this principal job?

B9. How many months does this salary cover?

B10. What will be your primary and secondary work activities?

Part C- Background Information

C1. Are you [male/female]

C2. What is your marital status?

C3. Not including yourself or your spouse/partner, how many dependents do you have?

C4. What is the highest educational attainment of your mother and father?

C5. What is your place of birth?

C6. What is your date of birth?

C7. What is your citizenship status?

C8. Of which country are you a citizen?

C9. In what state or country was the high school/secondary school

C10. Are you a person with a disability?

C11. Which of the following categories describes your disability?

C12. Are you Hispanic or Latino?

C13. Which of the following best describes your Hispanic origin or descent?

C14. What is your racial background?

C15. Please fill in the last 4 digits of your social security number.

C16. In case we need to clarify some of the information you have provided, please list an email address and telephone number where you can be reached.

C17. Please provide your address and the name and address of a person who is likely….

Back page:

Additions to Questions [A2 and A9]

Comments about the survey

Bar Codes and Survey Design

PROBES

ii. Probes: Consent language

[If respondent doesn’t appear to read the consent language]. I would like you to take a moment to locate the bottom two paragraphs on the front cover of the survey.

1. Did you read these two paragraphs before you started responding to the survey? Why/Why not?

If not--[Give respondent time to read]

2. Using your own words, can you please summarize what the first paragraph is saying?

3. What was your initial reaction to these paragraphs when you read them for the first time?

4. Do you feel that the wording of the first paragraph explains fully the way in which the information you provide will be used?

5. Do you feel that the information you provide will be secure and remain confidential?

6. Do you have any concerns about providing the last 4 digits of your social security number?

A2 Probes: Interdisciplinary

I would like to ask you a couple of questions about question A2.

1. Please describe your thought process as you answered this question.

2. Using your own words, could you please tell me the definition of “interdisciplinary?”

3. Do you think the term interdisciplinary will be clear to graduating doctoral students?

4. Was your dissertation research interdisciplinary? [If ‘YES’, continue]

5. Do you feel this question is worded such that it can easily accommodate respondents whose dissertations research was interdisciplinary?

6. What are your thoughts on the instructions in italics? Did you notice them? Are they clear?

A4. Probes: Remission

1. Please take a look at this question and rephrase it using your own words.

2. What is your understanding of the word “remission”?

3. Is there another synonym that you can think of that may be more clear and has the same meaning as “remission?”

4. As you were answering this question, what were your thoughts on what would be considered a “tuition remission”?

A8. Probes: Time to Degree

Please look at the series of questions at A8.

1. What were your initial thoughts and impressions as you completed this question?

2. What are your thoughts of the layout for this question? Was it clear to you as you answered each itemDo you have any suggestions?

3. Please tell me your thought process as you answered question item A8c. Did you have any difficulty How comfortable did you feel recollecting the month and year that you started each degree?

4. Further down the page at question A11, please tell me your thought process as you answered this question. [INTERVIEWER: Ask the same probe for questions A12 and A13].

B8 Probes: Salary response categories

I would like to ask you a couple of follow up questions regarding question B8.

1. How do you feel about providing your salary information? Is this something you are comfortable sharingHow comfortable are you about sharing it?

2. This question offers two ways to report your salary. When answering this question, did you provide the exact salary amount, or select a range? Why?

3. Do you feel there is a more convenient way to report salary or is this clear and easy to reportinformation? [Probe more if needed: Hourly? Weekly? Bi-monthly? Monthly? Etc.?]

4. Think back to when you were filling out the SED questionnaire at the time of your graduation. Do you recall whether you wWere you negotiating offers at that time?

[If YES]: Since you were negotiating between different offers, how do you think you would have answered this question?

5. What do you think the question is referring to when it says “basic annual salary?” [Try to get an idea of what they may have included when they thought about their total – are they including just salary or other parts of their package?]

6. In thinking about the salary you had in mind when you answered this question, where were you drawing your information from? [Try to get an idea of what the source of the knowledge is. Is it a contract, verbal agreement, something they saw online or talked about with friends, a guess?]

Survey Layout Probes: Reactions to Scannable Form and New Design

Now I would like to ask you about the layout of the questionnaire.

1. Overall, what was your initial impression regarding the layout?

2. [Ask respondent to look at both designs of the form] Comparing both forms page by page, please tell us your thoughts as you flip through each.

3. [If needed] How do you feel about the newer design for the “flip out” page of degree types? Which do you feel is more convenient?

4. When you were completing the survey, did you notice the bar code and the scanning marks?

1. [If R noticed the marks] What do you think the markings are for?

Did you have any concerns thoughts about these markings while completing the survey?

2. [If R did not notice the marks] Now that you are aware of the markings, What do you think the markings are for? Do you have any concerns thoughts about the survey because of these markings?

5. What is your overall understanding of the scanning process?

6. [If R mentions any concern about confidentiality in relation to scanning] Do you feel data are more or less confidential and secure if they are scanned into a database?

11