NIST, MEP Workcred Virtual Focus Group Collection - Supporting Statement

OMB Control No. # 0693-0033 – NIST Generic Clearance for Program Evaluation Data Collections

TITLE OF THE COLLECTION: NIST, MEP, Workcred Virtual Focus Group Collection

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

The National Institute of Standards and Technology (NIST) through the Hollings Manufacturing Extension Partnership (MEP) is conducting this research study to determine the quality, market value, and effectiveness of the credentials used in manufacturing as perceived by individuals responsible for workforce hiring and promotion decisions. A first phase of the project was a survey of small and medium sized manufacturing companies distributed across the United States and Puerto Rico, requesting that individuals with hiring authority (e.g., owner/CEO, presidents, chief operating officers, HR, etc.) complete the survey. 314 surveys were completed and provided information about what credentials are being used, why and why not.

As result of the survey data analysis, the project desires to conduct follow-up discussions with samples of the survey respondents. To minimize time and costs and to make the opportunity available for respondents, the project team will conduct virtual focus groups, using web-based technology. These focus group meetings will help clarify how companies value credentials, effectiveness, how value and effectiveness are measured, and how the credentials are being used in their particular work setting. The potential participants will be invited by the MEP centers, identified based on national survey response demographics, and survey response contact information.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The virtual focus group meeting questions are developed by the contractor, working with the NIST MEP Program Manager, Mary Ann Pacelli. Using the initial survey data, and matching results to the set of Research Goals initially defined during the planning and design of the survey project, the project team has generated a set of questions appropriate for group settings. The questions were reviewed by the initial pilot review team of 8 representatives from MEP centers across the U.S. Initial questions have been revised based on input from the review team and with the expertise of the research team, considering the meeting format and time constraints.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

The project team will schedule ten (10) 1-hour web-based meetings. The sessions will be scheduled over a 3-week period, with varying times to meet the schedules of 4 time zones of the U.S. An email invitation (uploaded into ROCIS) will be provided to the MEP Centers to use with their contact lists. The invitation will include all the available dates and times. Each session will have a unique registration link. When the individuals register, additional information about themselves is requested on the registration screen. Registration is required to participate in any of the focus groups. Sessions will be limited. Ideal participation is 10- 12, we will allow up to 15 registrations for each session, as experience with similar events shows that there could be a 10% or more no-show. If a registrant should exceed the 15 maximum, they will receive an email directing them to other available sessions. After registration, the individual will receive an initial email confirming their request that will include a notice that the events will be recorded. If the potential participant does not agree to be recorded, they will be guided to cancel registration. Within 24 hours they will receive registration confirmation that provides the web link information to participate. Two (2) days prior to the scheduled session, participants will receive an email with a copy of the collection instrument, NIST-MEP Workcred Focus Group Orientation Document and Protocol Discussion Guide, with the questions to be discussed during the virtual meeting.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The Focus Group Meetings will be facilitated by the NIST MEP project team using the designed NIST-MEP Workcred Virtual Focus Group Orientation and Protocol Discussion Guide. The virtual sessions will be recorded, with participant approval determined at registration. (see response in #3 above). The results of the focus group discussions will be analyzed by the Workcred project staff in partnership with NIST MEP Program Manager and merged with the initial survey data. During the planning and design phase, the project team defined and confirmed a set of Research Goals that guided the development of the survey instrument:

1. In general, are credentials valued?

2. In general, are credentials used by manufacturing?

3. Do they make a difference?

If yes, what are your reasons?

1. Productivity increases

2. Motivation increases

3. Retention increases

4. Promotions occur more quickly

If no, what are your reasons credentials are not used and valued?

1. Have to be retrained anyway

2. Don’t have the knowledge and skills that the credential states

3. No difference in performance between certified and non-certified

4. Experience is a better predictor for successful performance

5. Increase in salary associated with the credential is not cost effective

6. More cost effective to create our own credentials (or training program)

7. Has knowledge but can’t perform

4. At what job levels within manufacturing are credentials more effective?

1. Entry Level (new hire with minimally qualifications)

2. Midcareer

3. Senior level

5. In general, how are credentials used by the manufacturer?

• Preferred

• Required for employment

• Required for promotion

• Required for retention

• Not used

6. How do employees in manufacturing facilities assess potential hires to determine if they are qualified to competently perform at their jobs? What credentials are being used in the technical operations of manufacturing?

• This would include degrees, certificates, certifications and licenses, union credentials (apprenticeships)

7. Which credentials are the most prevalent?

• Across all manufacturing regardless of type of manufacturing

• According to type of manufacturing

• Related to state and geographic region

• According to size of manufacturing

8. What are the specific names of the credentials used within each type of job role associated with the technical operations of manufacturing?

9. What specific credentials are required by most manufacturers and why do they require them?

   1. Size of manufacturer

   2. Geographic region

   3. Sector of manufacturer

10. What specific credentials are generally preferred by most manufacturers and why do they prefer them?

11. When are the credentials generally acquired?

• Before hiring

• After they are hired

• Voluntarily obtained credential

• Needed to obtain a promotion

• Use as a professional development tool

12. What support are the workers given after they are hired to obtain a credential?

13. When individuals are hired without credentials, what type of training do you conduct to up-skill these employees?

14. When individuals are hired with credentials, do you still have to conduct training to up-skill employees?

MEP will use the initial survey results to explore various relationships to answer key questions of the study. The analysis will range from simple analysis such as frequency distributions, central measures and variance analyses (e.g., mean, mode, medians, standard deviation), and advanced analyses (correlations, regressions, t-tests, and ANOVAs). The analyses will help MEP answer questions that the study seeks to answer.

Responses will be collated and compared across a variety of factors including type of manufacturing sector, geographic region, type of job, role of respondent. This will allow MEP to identify key factors that influence the use and perceived of credentials and what employers are looking for in making workforce decisions.