The author reviews methods for developing a quality management system training presentation and exam to demonstrate training effectiveness for QMS procedures.
Training records are a basic expectation for any quality system (i.e., Clause 6.2.2 of ISO 13485), but demonstrating effectiveness (Sub-Clause 6.2.2c) and competency (Sub-Clause 6.2.2a) is also required (http://bit.ly/21CFR820-25 and http://bit.ly/13485Plus). Verifying training competency will be the subject of a future blog, but this blog focuses on the most common method for demonstrating training effectiveness—an exam.
The challenge with creating a training exam is that it takes a serious time commitment to create a training presentation, to write an exam, to grade the exam each time an employee is trained and issue training certificates. Each time you revise the procedure, you need to decide if re-training is required, and how you will verify effectiveness of training on the revised procedure. Finally, you need qualified trainers with appropriate documentation of their competency.
The larger your company is, the more value you will realize from creating training exams. Unfortunately, larger companies usually have more procedures too. It typically takes me 2-4 hours to develop a new training presentation for a process or procedure. I am currently developing training presentations for a small drug/device company that will have approximately 30 procedures. Therefore, it could take 60-80 hours to create training presentations for all the procedures.
Quality Management System Training Presentation Content
We have developed training courses for critical processes, such as CAPA, internal auditing and design controls. Medical Device Academy will also be offering a free webinar in November on the topic of conducting more effective Management Review meetings. For other processes, such as calibration, we recommend the following step-by-step approach:
- Overview slide of requirements
- A slide for each sub-clause
- An example of how the procedure is applied
- An overview of the procedures’ key points
Our calibration procedure required a total of 11 slides—including a title slide and a slide for contact information. We also included a cross-reference to the applicable section of the procedure for each sub-clause of the ISO 13485 Standard (i.e., 7.6a, 7.6b, etc.). A balance used to weigh components was the training example, and there was a slide that explained important considerations that can affect the accuracy of a balance.
During the process of creating the training presentation, we noticed that one of the sub-clauses of ISO 13485, 7.6, was not addressed by the procedure. Therefore, our systematic approach simplified the creation of a training presentation, and the approach helped to identify a nonconformity in the procedure before it was released.
For training exams, we try to ensure that exam questions are objective and easy to grade. Therefore, most exams are 10 questions. Questions are typically fill-in the blank type of questions or multiple choice questions. “Trick questions” are not recommended, but we do recommend using questions that force the trainee to look-up the answers. This will force the trainee to become more familiar with the procedure.
For our calibration training exam, we could easily have one question corresponding to each slide, but not every sub-clause requirement is equally important. Therefore, we always try to include a question related to the most common things auditors and FDA inspectors will be looking for.
The calibration process includes a requirement to assess the impact on product if a measurement device used for inspections is found to be Out-Of-Tolerance (OOT). Is there a need to retest or even recall product?
Almost every auditor I meet asks the above question during an audit, and most will even request records of calibrated devices that were found OOT.
Grading & Certificates
We typically use 70% as the criteria for passing an exam. The exams are protected forms with spaces to fill-in and check boxes. There are also spaces for the trainee’s name, title and date of training. Exams are emailed to the instructor, and the instructor will unprotect the document. The correct answers are indicated by highlighting the choice in green. Incorrect answers are indicated by highlighting the choice in red. Explanations for why an answer is incorrect are added after the question and highlighted in yellow. The graded exam is then emailed back to the trainee—along with a training certificate similar to the one shown at the beginning of this blog.
Brigid Glass wrote a blog posting recently (http://bit.ly/12PartSOPTemplate) where she recommended including a section on training and re-training requirements in each procedure. The challenge with revised procedures is that re-training is not always required. If re-training is needed, we recommend the following approach:
- Create a separate retraining exam
- Include a question(s) specific to the procedural revisions
- Include a question(s) specific to recent nonconformities in the process (if any)
If you are interested in learning more about procedures, records and training as it relates to ISO 13485 Certification, Brigid Glass and I will be teaching a seminar hosted by FX Conferences on November 5. The following link includes a $50 discount code for the seminar: http://bit.ly/ISOCertStep4. This is part of a 6-part ISO Certification series that is available as recordings too.