Internal Auditor Archives

Seven ways to improve quality auditor training

2 Comments / ISO Auditing / By Robert Packard / Internal Auditor, Train the trainer, Training

A five-day lead auditor course is never enough. Effective quality auditor training must include practical feedback from an expert.

What is required for quality auditor training?

The key to training auditors to audit is consistent follow-up over a long period of time (1-2 years, depending upon the frequency of audits). I recommend following the same training process that accredited auditors must complete. I have adapted that process and developed seven (7) specific recommendations.

Training the trainer

One of my clients asked me to create a training course on how to train operators. I could have taught the operators myself, but so many people needed training that we felt it would be more cost-effective to train the trainers. Usually, I have multiple presentations archived that I can draw upon, but this time I had nothing. I had never trained engineers on how to be trainers before—at least not formally. I thought about the problems other quality managers have had in training internal auditors and how I have helped the auditors improve. The one theme I recognized was that effective quality auditor training needs to include practical feedback from an experienced auditor. An expert auditor that is training new auditors needs to identify systematic ways to provide feedback, and setting a benchmark for the number of times feedback will be provided is really helpful.

Improve by observing yourself and other quality auditors

Observing someone else is a great way to learn when you are learning any new skill. Interns often do this, which is also a technique used to train new auditors. This technique is called shadowing. You can learn by watching, but eventually, you need to try to do tasks that are beyond your comfort level, and it is best to practice auditing with an expert watching you.

Practice team member audit preparation

Many of the internal auditing procedures we see require new auditors to conduct three audits as team members before they can audit independently. In contrast, notified body auditors join as team members for 10-20 audits before they can act as lead auditors. During the training period, auditors in training observe multiple lead auditors and multiple quality systems. Each audit allows auditors in training to write nonconformities and receive feedback from a lead auditor. At the beginning of quality auditor training, the focus must be on audit preparation. What are the areas of importance, what are the results of previous audits, are there any previous audit findings to close, etc? This preparation can even be done as practice for a hypothetical audit.

During quality auditor training, practice the opening and closing meetings

Opening and closing meetings are one of the first things to teach a new lead auditor. Have new lead auditors rehearse their first few opening and closing meetings with you in private before conducting the opening and closing meetings. Ensure the lead auditor has an opening/closing meeting checklist to help them. Recording practice sessions is enormously helpful because the trainee can watch and observe their mistakes. As trainees get more experience, the opening and closing meetings should have time limits. Finally, you might ask members of top management to challenge the lead auditor with questions. The lead auditor needs to be comfortable with their decisions and the grading of the audit findings.

How to practice audit team leadership

Have new lead auditors conduct team audits with another qualified lead auditor for 10-20 audits before you allow them to conduct an audit alone. Leading the opening and closing meetings is usually the first area new lead auditors master. The most complicated area to learn is managing a team of auditors. Team members will fall behind schedule during audits, or someone will forget to audit a process. As a lead auditor, you must complete the audits for your assigned processes and communicate with the entire team to ensure everyone else is on schedule. As an observer, you must let lead auditors make mistakes and help them realize them. Initially, a trainer will encourage new lead auditors to give themselves more than enough time. As their training progresses, the timing needs to be shorter and more challenging. Ultimately, you have to push the team beyond its capability to teach new lead auditors to recognize problem signs and teach them how to fix the problems.

Shadow auditors virtually with recordings

Live shadowing is challenging for experts and trainees because you are distracted by listening to the auditee and observing the auditor. However, if an audit is recorded, the person shadowing can watch the recording. The audit is already completed, and there is little need to concentrate on the auditee. A recording allows the observer to focus on the auditor. If a new auditor is conducting their first audit, an expert should shadow the trainee for 100% of the audit. Gradually the observation can decrease with each subsequent audit.

Practice note-taking with recorded audits

Taking detailed notes is something that experts take for granted, but I learned a lot by watching FDA inspectors take notes during an inspection. Have a new auditor observe a few audits before they are allowed to participate. Make sure they take notes and explain what you are doing and why they are observing as you conduct audits. Review the notes of new auditors periodically throughout the audit to provide suggestions for improvement and identify missing information. You can also record a supplier audit or internal audit and let a new trainee take notes on the pre-recorded webinar. This eliminates the need to coordinate schedules to involve the trainee.

Quality auditor training should include practicing audit agenda creation

Have new lead auditors submit a draft audit agenda to you before sending it to the supplier or department manager. Usually, the first audit agenda will need revision and possibly multiple revisions. Make sure you train the person to include enough detail in the agenda, and using a checklist or template is recommended. The agenda creation will be part of the audit preparation, and it can be done without time pressure.

How do you audit the auditing process?

Most quality managers are experienced and have little trouble planning an audit schedule. The next step is to conduct the audit. The problem is that there is very little objective oversight of the auditing process. The ISO 13485 standard for medical devices requires that “Auditors shall not audit their own work.” Therefore, most companies will opt for one of two solutions for auditing the internal audit process: 1) hire a consultant or 2) ask the Director of Regulatory Affairs to audit the internal auditing process.

Both of the above strategies for auditing the internal audit process meet the requirements of ISO 13485, but neither approach helps to improve an internal auditor’s performance. I have interviewed hundreds of audit program managers over the years, and the most common feedback audit program managers give is “Change the wording of this finding” or “You forgot to close this previous finding.” This type of feedback is related to the report-writing phase of the audit process. I rarely hear program managers explain how they help auditors improve at the other parts of the process.

When auditors are first being trained, we typically provide examples of best practices for audit preparation, checklists, interviewing techniques, AND reports. After auditors are “shadowed” by the audit program manager for an arbitrary three times, the auditors are now miraculously “trained.” Let’s see if I can draw an analogy to make my point.

That kind of sounds like watching your 16-year-old drive the family car three times and then giving them a license.

About the Author

Robert Packard is a regulatory consultant with 25+ years of experience in the medical device, pharmaceutical, and biotechnology industries. He is a graduate of UConn in Chemical Engineering. Robert was a senior manager at several medical device companies—including the President/CEO of a laparoscopic imaging company. His Quality Management System expertise covers all aspects of developing, training, implementing, and maintaining ISO 13485 and ISO 14971 certifications. 2009-2012, he was a lead auditor and instructor for one of the largest Notified Bodies. Robert’s specialty is regulatory submissions for high-risk medical devices, such as implants and drug/device combination products for CE marking applications, Canadian medical device applications, and 510(k) submissions. The most favorite part of his job is training others. He can be reached via phone at 802.258.1881 or by email. You can also follow him on LinkedIn, Twitter, and YouTube.

An Auditor’s Best Practices in Issuing a Major Nonconformity

4 Comments / ISO Auditing / By Robert Packard / Audit, Internal Auditor, ISO

From the opening meeting through the audit and closing meeting, the author describes an auditor’s best practices in issuing a major nonconformity.

As an auditor, one of the most important (and difficult) things to learn is how to issue a nonconformity—especially a major. This is usually done at the closing meeting of an audit, but the closing meeting is not where the process of issuing the nonconformity begins. Issuing a nonconformity starts in the opening meeting.

ISO 19011:2011 is the official guidance document for auditors of Quality Management Systems, and ISO 13485 is the quality system standard for medical device manufacturers. Section 6.4.2 of this Standard explains best practices for an opening meeting. The last five items in this section are critical to preparing the client for potential nonconformities:

Method of reporting audit findings, including grading, if any
Conditions under which the audit may be terminated
Time and place of the closing meeting
How to deal with possible findings during the audit
System for feedback from the auditee on findings or conclusions of the audit
Process for complaints and appeals

Methods of Reporting and Grading Nonconformities

The auditor should be crystal clear in their description of minor and major nonconformities or any other grading that will be used. The auditor should also make it clear that they are looking for conformity rather than nonconformity. This is an audit—not an inspection. Typically, a minor nonconformity is described as “a single lapse in the fulfillment of a requirement,” while a major nonconformity is described as one of the following: 1) “a total absence in the fulfillment of a requirement,” 2) “repetition of a previous nonconformity,” 3) “failure to address a previously identified minor nonconformity,” or 4) “shipment of non-conforming product.” When the auditor is in doubt, then the finding is minor, and never a major. For a major nonconformity to be issued, there can be no doubt.

Conditions for Termination

The option to terminate an audit is typically reserved for a certification audit where a major nonconformity is identified, and there is no point in continuing. Termination is highly discouraged, because it is better to know about all minor and major nonconformities right away, instead of waiting until the certification audit is rescheduled. The certification body will charge you for their time anyway.

Another reason for termination is when an auditor is unreasonable or inappropriate. This is rare, but it happens. If the audit is terminated, you should communicate this to upper management at the certification body and the company—regardless of which side of the table you sit. For FDA inspections, this is not an option. For audits performed by Notified Bodies, there is the possibility of suspension of a certificate in response to audit termination. Therefore, I always recommend appealing after the fact, instead of termination. Appealing also works for FDA inspections.

How to Deal with Findings

All guides and auditees should be made aware of possible findings at the time an issue is discovered. This is important so that an auditee has the opportunity to clarify the evidence being presented. Often, nonconformities are the result of miscommunication between the auditor and the auditee. This frequently happens when the auditor has a poor understanding of the process being audited. It is a tremendous waste of time for both sides when this occurs. If there is an actual nonconformity, it is also important to gather as much objective evidence as possible for the auditor to write a thorough finding and for the auditee to prepare an appropriate corrective action plan in response to the discovery.

Feedback from the Auditee

As an auditor, I always encourage auditees to provide honest feedback to me directly and to management, so that I could continue to improve. If you are giving feedback about an internal auditor or a supplier auditor, you should always give feedback directly before going to the person’s superior. You are both likely to work together in the future, and you should give the person every opportunity to hear the feedback first-hand.

When providing feedback from a third-party certification audit, you should know that there will be no negative repercussions against your company if you complain directly to the certification body. At most, the certification body will assign a new auditor for future audits and investigate the need for taking action against the auditor. In all likelihood, any action taken will be “retraining.” I never fired somebody for a single incident—unless they broke the law or did something unsafe. The key to providing feedback, however, is to be objective. Give specific examples in your complaint, and avoid personal feelings and opinions.

Complaints and Appeals

As the auditee, you should ask for the contact information of the certification body during the opening meeting. Ask with a smile—just in case you disagree, and so you can provide feedback (which might be positive). As the auditor, you should always make contact information for the certification body available. If you are conducting a supplier audit or an internal audit, you probably know the auditor’s boss, and there is perhaps no formal complaint or appeals process. In the case of a supplier audit, the customer is always right—even when they are wrong.

During the Audit

During the audit, you should always make the guide(s) and process owner(s) aware of any potential nonconformities as you find them. This is their opportunity to clarify the objective evidence for you and to explain why there is not a nonconformity. Often, at this point in the audit, I will refer to the Standard. I will identify the specific requirement(s) and show the process owner. I will say, “This is what I am trying to verify. Do you have anything that would help address this requirement?” If the process owner is unsure of how to meet the requirement, often, I will provide an example of how this requirement is addressed in other areas or at other companies.

If the audit is a multi-day audit, I will review the potential nonconformities at the end of the day and allow the auditee to provide additional objective evidence in the morning. If it is the last day of the audit, or it is a single-day audit, I will give auditees until the closing meeting to provide the objective evidence. Often, I will use this opportunity to explain what would be considered a minor nonconformity and what would be a major nonconformity. Usually, I can say, “This is not a major nonconformity because…”

Closing Meeting

The closing meeting should be conducted as scheduled, and the time/location should be communicated to upper management in the audit agenda and during the opening meeting. Top management won’t be happy about nonconformities, but failure to communicate when the closing meeting will be conducted will irritate them further.

At the closing meeting, the auditee should never be surprised. If an issue remains unfulfilled at the closing meeting, the auditee should be expecting a minor nonconformity—unless the issue warrants a major nonconformity. Since a minor nonconformity is described as “a single lapse in the fulfillment of a requirement,” it is difficult for an auditee to argue that an issue does not warrant a minor nonconformity. Typically, the argument is that you are not consistent with other auditors. The most common response to that issue is, “Audits are just a sample, and previous auditors may not have seen the same objective evidence.” The more likely scenario, however, is that the previous auditor interprets requirements, instead of reviewing requirements with the client, and ensuring both parties agree before a finding is issued.

If a finding is major, the auditee should have very few questions. Also, I often find the reason for a major nonconformity is a lack of management commitment to address the root cause of a problem. Issuing a major nonconformity is sometimes necessary to get management’s attention.

Regardless of the grading, all audit findings will require a corrective action plan—even an FDA warning letter requires a CAPA plan. Therefore, a major nonconformity is not a disaster. You just need to create a more urgent plan for action.