Methods for developing a quality management system training presentation and exam to demonstrate training effectiveness for QMS procedures.

Quality Management System Training

Training records are a basic expectation for any quality system (i.e., Clause 6.2.2 of ISO 13485), but demonstrating effectiveness (Sub-Clause 6.2.2c) and competency (Sub-Clause 6.2.2a) is also required. Verifying training competency will be the subject of a future blog, but this blog focuses on the most common method for demonstrating training effectiveness—an exam.

Resource Needs

The challenge with creating a training exam is that it takes a serious time commitment to create a training presentation, to write an exam, to grade the exam each time an employee is trained, and issue training certificates. Each time you revise the procedure, you need to decide if retraining is required and how you will verify the effectiveness of training on the revised procedure. Finally, you need qualified trainers with appropriate documentation of their competency.

The larger your company is, the more value you will realize from creating training exams. Unfortunately, larger companies usually have more procedures too. It typically takes me 2-4 hours to develop a new training presentation for a process or procedure. I am currently developing training presentations for a small drug/device company that will have approximately 30 procedures. Therefore, it could take 60-80 hours to create training presentations for all the procedures.

Quality Management System Training Presentation Content

We have developed training courses for critical processes, such as CAPA, internal auditing, and design controls. Medical Device Academy will also be offering a free webinar in November on the topic of conducting more effective Management Review meetings. For other processes, such as calibration, we recommend the following step-by-step approach:

1. Overview slide of requirements
2. A slide for each sub-clause
3. An example of how the procedure is applied
4. An overview of the procedures’ key points

Our calibration procedure required a total of 11 slides—including a title slide and a slide for contact information. We also included a cross-reference to the applicable section of the procedure for each sub-clause of the ISO 13485 Standard (i.e., 7.6a, 7.6b, etc.). A balance used to weigh components was the training example, and there was a slide that explained essential considerations that can affect the accuracy of a balance.

During the process of creating the training presentation, we noticed that one of the sub-clauses of ISO 13485, 7.6, was not addressed by the procedure. Therefore, our systematic approach simplified the creation of a training presentation, and the strategy helped to identify a non-conformity in the procedure before it was released.

Exam Content

Verifying effective training is easiest with the use of quizzes or exams to test what the person learned. For training exams, we try to ensure that exam questions are objective and easy to grade. Therefore, most exams are ten questions. Questions are typically the fill-in-the-blank type of questions or multiple-choice questions. “Trick questions” are not recommended, but we do recommend using questions that force the trainee to look up the answers. This will force the trainee to become more familiar with the procedure.

For our calibration training exam, we could easily have one question corresponding to each slide, but not every sub-clause requirement is equally important. Therefore, we always try to include a question related to the most common things auditors and FDA inspectors will be looking for.

The calibration process includes a requirement to assess the impact on a product if a measurement device used for inspections is found to be Out-Of-Tolerance (OOT). Is there a need to retest or even recall products?

Almost every auditor I meet asks the above question during an audit, and most will even request records of calibrated devices that were found OOT.

Grading & Certificates

We typically use 70% as the criteria for passing an exam. The exams are protected forms with spaces to fill in and checkboxes. There are also spaces for the trainee’s name, title, and date of the training. Exams are emailed to the instructor, and the instructor will unprotect the document. The correct answers are indicated by highlighting the choice in green. Incorrect answers are indicated by highlighting the choice in red. Explanations for why an answer is incorrect are added after the question and highlighted in yellow. The graded exam is then emailed back to the trainee—along with a training certificate similar to the one shown at the beginning of this blog.

Retraining

In one of our blog posts, we recommended including a section on training and retraining requirements in each procedure. The challenge with revised procedures is that retraining is not always required. If retraining is needed, we recommend the following approach:

1. Create a separate retraining exam
2. Include a question(s) specific to the procedural revisions
3. Include a question(s) specific to recent nonconformities in the process (if any)

If you are interested in learning more about procedures, records, and training as it relates to ISO 13485 Certification, please consider our ISO 13485 webinar training.