Cognitive Research for Potential New Space Data Points for the Survey of Science and Engineering Research Facilities

Appendix C: Interview Protocol

Protocol for Research Interviews

Survey of Science and Engineering Facilities

Part 1 – Research Space

Thank you for meeting with us today. The Survey of Science and Engineering Research Facilities is conducted every two years. Before each cycle, we examine the survey content to see if updates are needed, given changes to the way research is conducted.

Therefore, we are interviewing a number of universities and experts to get a better understanding of the usefulness of the current research space data. We’re also interested in knowing whether there are other data points we should be collecting and whether you think data for these potential areas are collectible. Before we begin, do you have any questions about the survey or about this session?

A few things before we begin… This interview is voluntary, so you may end it at any time and you are not obligated to answer any questions. We will prepare a summary of all of our interviews. It would be very helpful if we could audio record the discussion to ensure we have accurate notes. The names and content of this interview, including the audio recording, will only be used for internal analysis and questionnaire development by the NSF survey staff and the NSF survey contractor. Is it ok to record the interview?

(THIS EXCHANGE IS RECORDED. CONTINUE RECORDING UNLESS THE INTERVIEWEE DECLINES TO BE RECORED: Thank you for agreeing to have this interview recorded. We will now begin the interview.)

Questions for person in data user/analyst role:

1. First, tell me a little about yourself and your role at [Institution name].

   1. (Probe to understand the person’s role as a data user/analyst.)

   2. What decision-making authority do you have regarding research space management and funding?

2. Were you familiar with the survey questionnaire and products (InfoBrief, published tables, WebCASPAR), before we sent them to you?

IF YES: Which ones had you seen? How did you come across these? (through your own research, from NSF notifications, co-worker recommendation?)

a. Have you ever run reports from WebCASPAR?

1. What types of fields did you use in your reports?

2. Did you run these to benchmark against other institutions? (probe for any other uses of the reports)

3. Did you use other NSF data in WebCASPAR, such as degree data or research expenditure data?

4. Did you run reports for multiple years?

b. Have you also used the published tables or InfoBriefs for analysis?

IF YES: i. How were they used and were they useful?

ii. Were the tables with national totals on research space by field (or highest degree or control) useful?

iii. Were the tables on individual institutions useful?

IF NO: Are you familiar with the Facilities Survey research space (Part 1) data and publications?

IF YES: Run through the questions above to probe their past use of these resources.

IF NO: Ask them to tell us about their analytical activities involving research space data and benchmarking?

Please take a look at the research space questionnaire for the next few questions

(FOR ALL OF THESE QUESTIONS, BE SURE TO PROBE FOR EXPLANATIONS.)

3. What sources do you use to analyze or benchmark research space data?

a. What data points from these sources are most useful?

b. Can you provide one or two examples of the ways you’ve used data from these sources?

4. Please look at question 2. How useful is it to have data by the science and engineering fields in the current list?

1. What would be more useful?

2. How do you categorize users of your space internally?

3. If you track space by S&E field, would it be useful to have finer fields that what is presented in question 2?

   1. For example: by the subfields of engineering such as chemical, civil, electrical or mechanical engineering?

   2. Are there similar examples you have for the science fields?

4. Question 6 asks institutions to rate the condition of their research space. Have you ever used these national data (in tables 3) from the set of statistical tables we provide? (the individual institutional data is confidential)

1. Probe for reasons of usage/non-usage.

2. Do you conduct your own research space condition assessments for internal reviews? (probe for examples of how they use the NSF or their internal data for analysis)

3. How accurate do you think these data are?

4. Please look at Question 10 on sources of project funding from federal government, state and local government, and institutional/other sources. Data from this question are published in FY 2011 tables 31 through 34.

a. Is this useful to you or not useful?

b. Would you say the level of detail on project funding is too high, too low or about right?

c. What would be more useful to you?

4. Questions 9, 13 and 17 collect data on new construction projects, planned construction projects and deferred construction projects by S&E field. How useful is it to have the data broken out in this way?

1. Have you used these data points? (How were they used?)

2. What would be more useful to you?

3. How do you categorize these types of projects internally?

4. Similarly, questions 7, 11 and 15 collect data on repairs and renovations. Have you found these data to be useful or not?

1. IF USEFUL: How were they used?

2. Are the breakdowns in question 15 for “projects included in your institutional plan” and “projects not included in your institutional plan” useful?

4. Overall, are there data points we didn’t discuss earlier that you find useful?

10. Are there questions we should consider adding to the survey? What are the gaps in the research space data? What do you wish you knew?

Probe for specific areas to add

1. Are these data points that your institution could provide?

2. Do you think these data points are commonly tracked by your peers?

11. Are there questions that you do not find useful? (Probe for reasons. Do they believe these reasons are shared by others in the field?)

12. Is there anything else about the Facilities Survey questionnaire, data, or publications that you’d like to comment on?

Those are all of the questions that I have. We really appreciate your time and candid responses. This is valuable information that will help us to evaluate and improve the Facilities Survey. Thank you again for your continued participation in the survey and your help today.

Questions for survey person in data provider role:

1. First, tell me a little about yourself and your role at [Institution name].

   1. Probe to understand the person’s role as a data provider.

      1. Do you create research space reports for internal analysis?

      2. What kind of reports?

      3. Who uses them and how are they used?

2. Were you familiar with the survey questionnaire and products (InfoBrief, published tables, WebCASPAR) before we sent them to you?

IF YES: Which ones had you seen or used?

How did you come across these? (Through your own research, from NSF notifications, co-worker recommendation?)

Please take a look at the research space questionnaire for the next few questions

(FOR ALL OF THESE QUESTIONS, BE SURE TO PROBE FOR EXPLANATIONS.)

3. Please look at question 2, which asks for the amount of research space by S&E field.

1. How did you go about answering this question?

2. How easy or difficult was it to answer this question?

3. On page 7, physical sciences is broken into two groups. If we broke other fields into smaller groups, how easy or difficult would it be to provide data?

4. Are you familiar with the NCES Classification for Instructional Programs (CIP codes)? Did you use the CIP to answer this question or not?

4. Please review Question 6 on page 9, which asks for condition or research space.

   1. How did you gather data for this question?

   2. How do you calculate these percentages? (internal audits, estimates)

   3. How easy or difficult was it to answer this questions?

5. Please review the instructions at the top of question 7 regarding project costs. (Interviewer also notes the guidelines on completion costs and parsing costs for shared fields.)

   1. Does this threshold of $250,000, which is used for questions on construction costs as well as repairs and renovations seem too high, too low or about right?

   2. Do you have any concerns with this threshold when populating questions in these areas?

   3. How do you allocate costs for shared space?

6. Please review the individual project form for Question 9 (pages 13-16).

   1. How do you identify the fiscal year in which the physical work for new construction began?

   2. How easy or difficult is it to answer this question?

   3. In 9C we ask for various components of the expected space for the projects. Do your records provide this level of detail or not? What sort of information is available to you? (probe for potential problems)

   4. Do you internally track the difference between the estimated costs and estimated square footage as compared to the final costs and final square footage?

      1. IF YES: How closely do they align?

      2. IF NO: How closely do think they align? (probe on reasons for perception of accuracy/inaccuracy)

7. Please look at question 10. This question asks for sources of project funding from federal government, state and local government, and institutional/other sources.

   1. How easy or difficult was it to answer this question?

   2. Would you say the level of detail on the project funding is too high, too low or about right?

   3. What would be more useful to you?

8. Please look at question 15. This question asks for estimated costs for deferred repairs and renovations.

   1. How do you assemble data for the costs for these fields?

   2. How easy or difficult is this process? (please describe)

   3. How accurate or inaccurate do you believe this approach is?

9. Please look at question 17. This questions asks for estimated costs for deferred new construction projects

   1. How do you assemble data for the costs for these fields?

   2. How easy or difficult is this process? (please describe)

   3. How accurate or inaccurate do you believe this approach is?

10. Are there any (other) questions on the survey that you think are hard to answer?

IF YES: Which questions? Why are they difficult?

11. Are there any questions on the survey that ask for information your institution doesn’t track? (i.e. don’t get requests for these data points for internal reports)

    1. If the person is an internal data provider, what are some the research space reports they provide internally?

12. Is there anything else about the Facilities Survey questionnaire, data, or publications that you’d like to comment on?

Those are all of the questions that I have. We really appreciate your time and candid responses. This is valuable information that will help us to evaluate and improve the Facilities Survey. Thank you again for your continued participation in the survey and your help today.

1