Supporting Statement

OMB Control No. # 0693-0033 – NIST Generic Clearance for Program Evaluation Data Collections

Examiner Preparation Course Evaluation Survey

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

Public Law 100-107 (The Malcolm Baldrige National Quality Improvement Act of 1987) that established the Baldrige Performance Excellence Program and its Malcolm Baldrige National Quality Award stipulates that organizational applicants for the award (see OMB Control #0693-0006) receive “an intensive evaluation by a competent board of examiners which shall review the evidence submitted by the organization and, through a site visit, verify the accuracy of the quality improvements claimed.”

These examiners receive training (depending on their levels of experience) annually. In 2020, 2021, and 2022, when travel came with concerns due to COVID, that training became virtual with shorter sessions spread over several weeks. Prior to then and to be determined in 2023 and beyond, that training may be in Gaithersburg, MD, over 3-4 days based on experience level. The intense training requires prework (the assessment of a fictitious applicant for the Malcolm Baldrige National Quality Award) and uses an adult-learning model that incorporates small- and large-group discussions, coaching, and other interactive exercises. This training has been offered for 30+ years, so it is important to ask the examiners after the training about its value. Examiners have expressed the desire to offer feedback after training.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The Examiner Preparation Course was developed 30+ years ago and improved as needed. To improve the course, focus groups of examiners and state Baldrige-based program directors have offered both formal and informal feedback, but a formal, written survey has been found to be the best way to uncover feedback from all examiners.

Suggestions for improvement are always sought and implemented as appropriate. Suggestions over the years have included the addition of a module on senior examiners coaching new examiners, a hotline for all examiners to ask questions prior to training, and online modules for examiners to complete prior to training.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

All ~300 examiners who complete the training will be surveyed via e-mail. The goal is for the survey to arrive no more than two weeks after training is complete and the examiners have returned home. The expected response rate is 75%, as examiners remain very engaged after training as they await their Baldrige Award process assignments.

To improve the response rate, examiners can be reminded during their assignments that their feedback can be used to improve training and the program. At the next year’s training, Baldrige staff have briefly pointed out to examiners that adjustments have been made to the training based on their feedback.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

Baldrige staff members who have extensive training in facilitation, adult learning, and organizational development will analyze the results to determine how training for the next year might be adjusted. Focus groups of examiners could also be pulled together if there is a major change proposed to training.