Supporting Statement

OMB Control #0693-0033

Expiration Date: 09/30/2025

NIST Generic Clearance for Program Evaluation Data Collections

Aspects of Postdoctoral Researcher Experience Scale NIST Survey

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

Background

High attrition rates in science, technology, engineering, and mathematics disciplines are an ongoing problem. Most studies to date have focused on undergraduate and graduate student attrition, with a particular emphasis on students’ attributes, undergraduate preparation, and mentoring relationships. Systematic studies looking into the experiences of Postdoctoral Researchers are few and far between. One example is a recent post-COVID survey organized by Nature. The responses to Nature’s survey uncovered a list of issues that Postdoctoral Researchers face: “high pressure, long hours, relatively low wages compared with those for analogous research positions outside academia, and pervasive job insecurity”. Another example is a study that looks into challenges that female postdocs face in academia (R. Ysseldyk et al. Front. Psychol. 10, 1297, 2019). The study revealed that a sense of belonging and the overall work environment are critical for female postdocs to overcome the decision to leave academia.

The goal of the proposed study is to assess the experiences of Postdoctoral Researchers at NIST. This study will be used to inform what changes might be necessary to ensure that the time Postdoctoral Researchers spent at NIST prepares them well for the next steps on their career path.

Target population

This survey will be administered to all Postdoctoral Researchers, including Federal and non-Federal employees. Non-Federal employees may include guest researchers, contractors, and other early-career researchers working at or for NIST. It is important to survey all Postdoctoral Researchers to ensure all viewpoints regarding experiences at NIST are considered and to capture an unbiased sample. For the purpose of this work, Postdoctoral Researchers are defined as early-career researchers no more than 7 years from receiving their Ph.D. This includes NRC postdocs, federal term employees, and guest researchers (US citizens and foreign nationals) supported via the PREP program or other agreement with external institutions.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The original “Aspects of Student Experience Scale” (ASES) survey was developed by researchers at Western Michigan University to study what factors that foster a supportive environment for graduate students. The questions included in ASES were developed based on prior literature and the American Physical Society Bridge Program (APSBP) recommendations, revised based on APSBP feedback, and subjected to psychometric evaluation. It has been demonstrated that ASES meets the standard criteria for divergent validity, discriminant validity, and internal consistency. The Aspects of Postdoctoral Researcher Experience Scale (APRES) Survey is a revised version of the ASES survey, modified to be relevant to Postdoctoral Researchers. The revisions include replacing references to the “department” with “OU” or “division”, as appropriate, replacing “graduate student” with “Postdoctoral Researcher”, “courses” with “professional development”, etc. The resulting APRES was discussed with researchers having prior experience working with ASES. We have made every attempt to ensure the resulting APRES survey is as anonymous as possible, while still ensuring we are able to capture data that will enable us to tease out where we may have concerns about the Postdoctoral Researchers' experiences at NIST. The analysis plan is described in question 4.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

The survey will be administered annually and electronically using Google Forms (or Qualtrics). The survey will be anonymous (i.e., no identifiable information such as email address or NIST aliases will be collected) and only aggregated data will be analyzed. An email invitation, provided below, will be sent to all Postdoctoral Scholars. Given NIST’s Postdoctoral Researchers total staff is estimated to be about 400 people, the potential burden for this survey is estimated at: 400 respondents * 15 (minutes) / 60 (minutes in an hour) = 100 burden hours. The expected response rate is approximately 40%. We intend to send follow-up reminders and encourage participation at staff meetings and town hall events.

Proposed survey invitation email:

Dear Postdoctoral Researchers,

We invite you to consider participating in a survey about your experiences as a Postdoctoral Researcher at NIST. The purpose of this survey is to assess how Postdoctoral Researchers find themselves at NIST and to identify areas where their experiences need improvements. The results of this survey will be shared with NIST leadership and will be used to inform training programs and policies.

For the purpose of this work, Postdoctoral Researchers are defined as early-career researchers no more than 7 years from receiving their Ph.D. This includes NRC postdocs, federal term employees, and guest researchers (US citizens and foreign nationals) supported via the PREP program or other agreement with external institutions.

No identifying information about you will be collected. Survey data will be summarized, and aggregated results will be included in presentations and publications which may be shared with NIST management, staff, and the public. The raw, individual data will be accessed only by the team administering this survey, which is led by Justyna Zwolak.

The survey is completely voluntary and should take you about 15-20 minutes to complete. If you have any questions, please contact Justyna Zwolak, Ph.D., at jpzwolak@nist.gov or 301.975.0527.

Your opinion is very important!

Thank you,

Justyna Zwolak

————————————————————————

Applied and Computational Mathematics Division

National Institute of Standards and Technology

100 Bureau Drive, Stop 8910

Gaithersburg, MD 20899-8910 USA

Phone: (301) 975-0527

Email:  jpzwolak@nist.gov

————————————————————————

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The survey results will be aggregated and analyzed to identify common themes in responses. These themes may include Postdoctoral Researchers’ social and scholarly exploration support, mentoring and research experience, professional and academic development, and financial stability. The proposed analysis plan is listed below. We will be looking for areas where we need to improve Postdoctoral Researchers' experiences at NIST. Finally, the results will be shared in an annual report to NIST leadership, staff, and the public. The results will be posted to NIST internal and external websites.

• Preliminary analysis

  • Quantitative: Summary results for each question

    • For multiple-choice questions: count per response

    • For scaled questions: mean response

• Secondary analysis

  • Analysis using the Network analysis approach to Likert-style surveys (NALS) of the full response data.

  • NALS analysis for response grouped by

    • Type of appointment (Federal employee and non-Federal employee)

    • Tenure at NIST

    • Site

    • Prior experience

    • Sex

    • Ethnicity (we may group ethnicity into minority and non-minority if the response rate does not allow a more detailed analysis)

  • Exploratory analysis: comparison between groups

• Future analyses will include comparing these results over time as well as between OU as we implement the survey annually.