Supporting Statement

OMB Control #0693-0043

Expiration Date: 03/31/2022

NIST Generic Clearance for Usability Data Collections

NIST SME Automation Challenges & Opportunities Survey

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

The Measurement Science for Manufacturing Robotics program at the National Institute of Standards and Technology aims to develop and deploy measurement science, standards, and test methods that advance manufacturing robotic system performance. To inform the research that the program conducts and ensure research outputs are appropriately targeted to benefit US manufacturers, we would like to survey manufacturing employees to learn about their current work challenges and opportunities and assess the state of automation within their facilities.

Results from this survey will be directly applied to the program’s projects and goals to better assist US manufacturers in accordance with the NIST mission.

We are working with the NIST Manufacturing Extension Partnership (MEP) and the Advanced Research for Manufacturing (ARM) Institute to 1) facilitate the distribution of emails to their constituents to invite participation in the survey, 2) ensure anonymity as we will not have access to the email lists and therefore not know who has been invited to participate or which partner organization forwarded the invitation, and 3) specifically target geographic regions and industries that are expected to benefit from advancements in automation.

Manufacturers with facilities based in the US will be surveyed due to their insider knowledge of challenges and opportunities with regards to automating production facilities. Both management and manufacturing floor employees will be surveyed to obtain a spectrum of opinions on automation needs. These two groups are specifically targeted given their unique perspectives of manufacturing operations. Management respondents are expected to provide insights of high-level operational challenges and opportunities that are targeted at growing the business. Floor-level employee respondents have first-hand experience with the equipment and processes, and therefore have specific perspectives of the issues regarding the technologies leveraged.

A better understanding of both groups’ needs will benefit NIST’s work on related topics and will result in measurement science innovations targeted to industry’s needs.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The survey was initially developed using internal NIST Engineering Laboratory knowledge of manufacturing processes and automation opportunities based on years of interactions with small- and medium-sized enterprises (SMEs). Prior interactions have been facilitated by on-site visits, direct research collaborations, industry forums and trade events, and standards activities. The developers of the survey additionally consulted with survey experts from NIST’s Communications Technology Laboratory; early drafts of survey questions and the surveying process were significantly refined following those consultations. These early drafts were also shared with NIST MEP management to illustrate the refinement process, provide context for the survey rationale, and secure our collaboration with NIST MEP.

After the survey was developed, a focus group was conducted with members of the ARM Institute’s Technical Advances Committee (TAC). The ARM Institute is a Manufacturing Innovation Institute and part of the Manufacturing USA network; it consists of about 300 members from organizations who work with robotics, automation, and manufacturing. The TAC drives technical advances related to manufacturing, including creating roadmaps for automation, which is directly relevant to the goals of this survey.

The focus group assisted with further refinement of the survey, including providing suggestions of additional, targeted questions to include on the survey, expanding the roles and operational duties of potential respondents, and suggesting edits to existing questions. Three members of the ARM TAC also participated in individual pilot studies, where they discussed specific survey questions with us during individual 30-minute calls. Feedback from these participants directly influenced the survey questions.

The survey was additionally presented to the NIST MEP’s Industry 4.0 Working Group. Their feedback was leveraged to validate the contents of the survey and the targeted demographics of respondents.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

Potential respondents will be contacted directly by email. To reach relevant respondents, we will leverage existing email lists from NIST MEP and the ARM Institute (mentioned in Q2). Emails will be disseminated via the respective organizations, and we will not have access to any personally-identifiable information (PII) or business identifiable information (BII) (including contact email addresses, names, or demographics). In particular, a selection of MEP centers representing areas across the country will be leveraged to disseminate the survey to their local member companies. Additionally, as a Manufacturing Innovation Institute, the ARM Institute has as members many manufacturing employees who are aware of the state of automation in their industry and can provide detailed responses to our survey.

The online survey will be conducted anonymously and electronically using the Qualtrics survey platform. The survey consists of several generalized multiple-choice questions intended to highlight potential areas where automation and emerging technologies could benefit company operations. At the end of the survey are a number of open-ended questions. Even though subsequent questions are based on prior answers, respondents are not required to answer any of the questions. Skipping certain questions will take respondents directly to the end of the survey. If they choose to do so, respondents may provide email addresses for follow-up.

Responses will be collected until either 1) 2,000 total responses have been received, or 2) the survey has been open for two months. The survey was designed to take approximately 10 minutes to complete. At maximum, we calculate a burden of 2000 responses * 10 minutes => 333.33 burden-hours. However, we expect a response rate of 25-30%. All responses received will be sampled.

To improve response rate but avoid over-burdening potential respondents, one reminder email will be sent two weeks prior to the survey closing date. Broad announcements may also be disseminated via NIST social media (e.g., the NIST blog, Twitter, and Facebook accounts) and specific industry forums (e.g., through the Association for Advancing Automation, Manufacturing USA, and the Material Handling Industry association). Such announcements are expected to raise awareness of the survey but are not expected to drive significant survey participation.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The survey consists predominantly of multiple-choice questions, which will be tallied to provide high-level insights into industry challenges and opportunities. Free-form questions will be collected and categorized based on prominent themes and specific content. Any PII or BII will be redacted. Email addresses and contact information, if provided, will be separated from the questionnaire responses for future follow-up discussions, but will not be disseminated.

Quantifiable results of the survey will be collated for public access and dissemination, and the results will be analyzed using statistical methods to test generalizability. Outliers and irregularly unique survey responses will be tagged and identified for further analysis. Statistical analysis will categorize responses into classes for identifying similarities and differences amongst sample groups. If this analysis is determined to be statistically generalizable, then the clustering of responses amongst survey results can be generalized for broad dissemination to the national SME population. Further analysis will be conducted to identify significant overall trends amongst the responses and to determine if the trends are generalizable to other segments of the population.

Qualitative results obtained from short-answer questions will be read by the survey team and grouped by similarity. Trends will be identified and noted in published results.