MEP - Resarch Examining the Quality, Markey Value and Effectiveness of Manufacturing Credentials - Four Standard Questions

OMB Control No. # 0693-0033 – NIST Generic Clearance for Program Evaluation Data Collections

TITLE OF THE COLLECTION: Research Examining the Quality, Market Value, and Effectiveness of Manufacturing Credentials

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

The National Institute of Standards and Technology (NIST) through the Hollings Manufacturing Extension Partnership (MEP) is conducting this research study to determine the quality, market value, and effectiveness of the credentials used in manufacturing as perceived by individuals responsible for workforce hiring and promotion decisions. A first phase of the project was a survey of small and medium sized manufacturing companies distributed across the United States and Puerto Rico, requesting that individuals with hiring authority (e.g., owner/CEO, presidents, chief operating officers, HR, etc.) complete the survey. A total of 314 surveys were completed and provided information about what credentials are being used, why and why not.

As result of the survey data analysis, the project will conduct in-person follow-up survey meetings in a focus group manner with small samples of the survey respondents. These focus group meetings will help clarify how companies value credentials, effectiveness, how value and effectiveness are measured, and how the credentials are being used in their particular work setting. The focus group meeting participants will be invited by the MEP centers, identified based on national survey response demographics.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The focus group meeting questions are developed by the contractor, working with the NIST MEP Program Manager, Mary Ann Pacelli. Using the initial survey data, and matching results to the set of Research Goals initially defined during the planning and design of the survey project, the project team has generated a set of questions appropriate for group settings. The questions were reviewed by the initial pilot review team of 8 representatives from MEP centers across the U.S. Initial questions have been revised based on input from the review team and with the expertise of the research team.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

The focus group meetings will be scheduled in the 10 states that provided the highest percentage of survey respondents: CA, SC, AZ, DE, FL, GA, ME, NJ, VT, WY. The MEP Centers in these states will invite small and mid-sized manufacturers located within a 2-hour drive time from their main office. The focus group meetings will be limited to a maximum of 12 participants per group. We estimate an average of 10 participants per meeting at each of the 10 meetings. The Focus Group Meeting questionnaire is designed to be completed in 2 hours or less. The MEP centers will invite companies with an email letter (sample attached) and request confirmed attendance via and email. MEP Centers will do a confirmation/reminder email and/or phone call to confirmed participants 3 days prior to the event to assure attendance. MEP centers will confirm at least 15 participants per meeting, as experience with similar events shows that there could be a 10% or more no-show.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The Focus Group Meetings will be facilitated by trained MEP Center staff who will be provided with a facilitator guide. The identified MEP Center facilitators will participate in a 2-hour live training provided via web link by the Workcred team. The training will review the purpose, expectations, and details of the focus group protocol guide. The MEP Centers will also provide a note taker in each session and will also record the sessions for the purpose of transcribing. The notes and meeting recorded transcription will be provided to the Workcred project team for analysis. The results of the focus group meetings will be analyzed by the Workcred project staff in partnership with NIST MEP Program Manager and merged with the initial survey data. During the planning and design phase, the project team defined and confirmed a set of Research Goals that guided the development of the survey instrument:

1. In general, are credentials valued?

2. In general, are credentials used by manufacturing?

3. Do they make a difference?

If yes, what are your reasons?

1. Productivity increases

2. Motivation increases

3. Retention increases

4. Promotions occur more quickly

If no, what are your reasons credentials are not used and valued?

1. Have to be retrained anyway

2. Don’t have the knowledge and skills that the credential states

3. No difference in performance between certified and non-certified

4. Experience is a better predictor for successful performance

5. Increase in salary associated with the credential is not cost effective

6. More cost effective to create our own credentials (or training program)

7. Has knowledge but can’t perform

4. At what job levels within manufacturing are credentials more effective?

1. Entry Level (new hire with minimally qualifications)

2. Midcareer

3. Senior level

5. In general, how are credentials used by the manufacturer?

• Preferred

• Required for employment

• Required for promotion

• Required for retention

• Not used

6. How do employees in manufacturing facilities assess potential hires to determine if they are qualified to competently perform at their jobs? What credentials are being used in the technical operations of manufacturing?

• This would include degrees, certificates, certifications and licenses, union credentials (apprenticeships)

7. Which credentials are the most prevalent?

• Across all manufacturing regardless of type of manufacturing

• According to type of manufacturing

• Related to state and geographic region

• According to size of manufacturing

8. What are the specific names of the credentials used within each type of job role associated with the technical operations of manufacturing?

9. What specific credentials are required by most manufacturers and why do they require them?

   1. Size of manufacturer

   2. Geographic region

   3. Sector of manufacturer

10. What specific credentials are generally preferred by most manufacturers and why do they prefer them?

11. When are the credentials generally acquired?

• Before hiring

• After they are hired

• Voluntarily obtained credential

• Needed to obtain a promotion

• Use as a professional development tool

12. What support are the workers given after they are hired to obtain a credential?

13. When individuals are hired without credentials, what type of training do you conduct to up-skill these employees?

14. When individuals are hired with credentials, do you still have to conduct training to up-skill employees?

We will use the initial survey results to explore various relationships to answer key questions of the study. The analysis will range from simple analysis such as frequency distributions, central measures and variance analyses (e.g., mean, mode, medians, standard deviation), and advanced analyses (correlations, regressions, t-tests, and ANOVAs). The analyses will help us answer questions that the study seeks to answer.

Responses will be collated and compared across a variety of factors including type of manufacturing sector, geographic region, type of job, role of respondent. This will allow us to identify key factors that influence the use and perceived of credentials and what employers are looking for in making workforce decisions.