Auditor shadowing as an effective auditor training technique

This article reviews auditor shadowing as an effective auditor training technique, but we also identify five common auditor shadowing mistakes.

How do you evaluate auditor competency?

Somewhere in your procedure for quality audits, I’ll bet there is a section on auditor competency. Most companies require that the auditor has completed either a course for an internal auditor or a lead auditor course. If the course had an exam, you might even have evidence of training effectiveness. Demonstrating training competence is much more challenging. One way is to review internal audit reports, but writing reports is part of what an auditor does. How can you evaluate an auditor’s ability to interview people, take notes, follow audit trails, and manage their time? The most common solution is to require the auditor “shadow” a more experienced auditor several times, and then the trainer will “shadow” the trainee.

Auditor shadowing in 1^st party audits

ISO 19011:2018 defines first-party audits as internal audits. When first-party auditors are being shadowed by a trainer or vice versa, there are many opportunities for training. The key to the successful training of auditors is to recognize teachable moments.

When the trainer is auditing, the trainer should look for opportunities to ask the trainee, “What should I do now?” or “What information do I need to record?” In these situations, the trainer asks the trainee what they should do BEFORE doing it. If the trainee is unsure, the trainer should immediately explain what, why, and how with real examples.

When the trainer is shadowing, the trainer should watch and wait for a missed opportunity to gather important information. In these situations, the trainer must resist guiding the trainee until after the trainee appears to be done. When it happens, sometimes the best tool is simply asking, “Are you sure you got all the information you came for?”

Here are five (5) mistakes that I observed trainers made when they were shadowing:

1. Splitting up, instead of staying together, is one of the more common mistakes I have observed. This happens when people are more interested in completing an audit rather than taking advantage of training opportunities. The trainee may be capable of auditing independently, but this is unfair to the trainee because they need feedback on their auditing technique. This is also unfair to the auditee because it is challenging to support multiple auditors simultaneously. When it is unplanned, trainers may not be available for both auditors. If an audit is running behind schedule, this is the perfect time to teach a trainee how to recover sometime in their schedule. Time management is, after all, one of the most challenging skills for auditors to master.

2. Staying in the conference room instead of going to where the work is done is a common criticism of auditors. If the information you need to audit can be found in a conference room, you could have completed the audit remotely. This type of audit only teaches new auditors how to take notes. These are necessary skills that auditors should master in a classroom before shadowing.

3. Choosing an administrative process is a mistake because administrative processes limit the number of aspects of the process approach that an auditor-in-training can practice. Administrative processes rarely have equipment that requires validation or calibration, and the process inputs and outputs consist only of paperwork, forms, or computer records. With raw materials and finished goods to process, the auditor’s job is more challenging because there is more to be aware of.

4. Not providing honest feedback is a huge mistake. Auditors need to be thick-skinned, or they don’t belong in a role where they will criticize others. Before you begin telling others how to improve, you must self-reflect and identify your strengths and weaknesses. Understanding your perspective, strengths, weaknesses, and prejudices is critical to being a practical assessor. As a trainer, it is your job to help new auditors to self-reflect and accurately rate their performance against objective standards.

5. “Silent Shadowing” has no value at all. By this, I mean shadowing another auditor without asking questions. You should mentally pretend you are doing the audit if you are a trainee. Whenever the trainer does something different from how you would do things, you should make a note to ask, “Why did you do that?” If you are the trainer, you should also mentally pretend you are doing the audit. It is not enough to be present. Your job is to identify opportunities for the trainee to improve. The better the trainee, the more challenging it becomes to identify areas for improvement.  This is why training other auditors have helped me improve my auditing skills.

Auditor shadowing in 2nd party audits

Auditors responsible for supplier auditing are critical to supplier selection, supplier evaluation, re-evaluation, and the investigation of the root cause for any non-conformities related to a supplier. Auditor shadowing is a great tool to teach supplier auditors and other people responsible for supply-chain management what to look at and what to look for when they audit a supplier. If you are developing a new supplier quality engineer responsible for performing supplier audits, observing the auditor during some actual supplier audits is recommended. Supplier audits are defined as second-party audits in the ISO 19011 Standard. The purpose of these audits is not to verify conformity to all the aspects of ISO 13485. Instead, the primary purpose of these audits is to verify that the supplier has adequate controls to manufacture conforming products for your company consistently. Therefore, processes such as Management Review (Clause 5.6) and Internal Auditing (Clause 8.2.2) are not typically sampled during a second-party audit.

The two most valuable processes for a second-party auditor to sample are 1) incoming inspection and 2) production controls. Using the process approach to auditing, the second-party auditor will have an opportunity to verify that the supplier has adequate controls for documents and records for both of these processes. Training records for personnel performing these activities can be sampled. The adequacy of raw material storage can be evaluated by following the flow of accepted raw materials, leaving the incoming inspection area. Calibration records can be sampled by gathering equipment numbers from calibrated equipment used by both processes. Even process validation procedures can be assessed by comparing the actual process parameters being used in manufacturing with the documented process parameters in the most recent validation or re-validation reports.

I recommend having the trainee shadow the trainer during the process audit of the incoming inspection process and for the trainer to shadow the trainee during the process audit of production processes. The trainee should ask questions between the two process audits to help them fully understand the process approach to auditing. Supplier auditors should also be coached on techniques for overcoming resistance to observing processes involving trade secrets or where competitor products may also be present. During the audit of production processes, the trainer may periodically prompt the trainee to gather the information that will be needed for following audit trails to calibration records, document control, or for comparison with the validated process parameters. The “teachable moment” is immediately after the trainee misses an opportunity, but while the trainee is still close enough to go back and capture the missing details.

Are you allowed to shadow a 3rd party auditor or FDA inspector?

Consider using 3rd party audits and inspections as an opportunity to shadow experienced auditors to learn what they are looking at and what they look for. In addition to shadowing an expert within your own company or an auditor/consultant you hire for an internal audit, you can also shadow a 3rd party auditor or an FDA inspector. This concept was the subject of a discussion thread I ran across on Elsmar Cove from 2005. The comments in the discussion thread supported the idea of shadowing a 3rd party auditor. The process owner (i.e., the manager responsible for that process) should be the guide for whichever process is being audited, and the process owner is responsible for addressing any non-conformities found in the area., The process owner should be present during interviews, but the process owner should refrain from commenting. The 3rd party auditor and the process owner need to know if the person being interviewed was effectively trained and if they can explain the process under the pressure of an audit or FDA inspection. If you are interested in implementing this idea, I recommend using one of two strategies (or both):

1. Consider having the internal auditor that audited each process shadow the certification body auditor for the processes they audited during their internal audit. This approach will teach your internal auditor what they might have missed, and they will learn what the 3rd party auditors look for to simulate a 3rd party audit more effectively when conducting internal audits.
2. Consider having the internal auditor that is assigned to conduct the next process audit of each process shadow the certification body auditor for that process. This approach will ensure that any nonconformities observed during the 3rd party audit are checked for the effectiveness of corrective actions during the next internal auditor. Your internal auditor will know precisely how the original nonconformity was identified and the context of the finding.