Auditor shadowing as an effective auditor training technique

This article reviews auditor shadowing as an effective auditor training technique, but we also identify five common auditor shadowing mistakes.

If you are shadowing, you are taking notes, so you can discuss your observations with the person you are shadowing later.

How do you evaluate auditor competency?

Somewhere in your procedure for “Quality Audits,” I’ll bet there is a section on auditor competency. Most companies require that the auditor has completed either a course for an internal auditor or a lead auditor course. If the course had an exam, then you might even have evidence for training effectiveness. Demonstrating competency is much harder. One way is to review internal audit reports, but writing reports are just part of what an auditor does. How can you evaluate an auditor’s ability to interview people, take notes, follow audit trails, and manage their time? The most common solution is to require that the auditor “shadow” a more experienced auditor several times, and then the trainee will be “shadowed” by the trainer. 

Auditor shadowing in 1^st party audits

ISO 19011:2011 defines first-party audits as internal audits. When first-party auditors are being shadowed by a trainer or vice versa, there are many opportunities for training. The key to the successful training of auditors is to recognize teachable moments.

When the trainer is auditing, the trainer should look for opportunities to ask the trainee, “What should I do now?” or “What information do I need to record?” In these situations, the trainer is asking the trainee what they should do BEFORE they do it. If the trainee is unsure, the trainer should explain what, why, and how at that moment with real examples.

When the trainer is shadowing, the trainer should watch and wait for a missed opportunity to gather important information. In these situations, the trainer must resist guiding the trainee until after the trainee appears to be done. When it happens, sometimes the best tool is simply asking, “Are you sure you got all the information you came for?”

Here are five (5) mistakes that I had observed trainers make when they were shadowing:

1. Splitting up, instead of staying together, is one of the more common mistakes I have observed. This happens when people are more interested in completing an audit, rather than taking every advantage of training opportunities. The trainee may be capable of auditing on their own, but this is unfair to the trainee because they need feedback on their auditing technique. This is also unfair to the auditee because it is challenging to support multiple auditors simultaneously. When it is unplanned, there may not be trainers available for both auditors. If an audit is running behind schedule, this is the perfect time to teach a trainee how to recover sometime in their schedule. Time management is, after all, one of the hardest skills for auditors to master.

2. Staying in the conference room, instead of going to where the work is done, is a common criticism of auditors. If the information you need to audit can be found in a conference room, then you could have completed the audit remotely. This type of audit teaches new auditors very little, other than how to take notes. These are necessary skills that auditors should master in a classroom before shadowing.

3. Choosing an administrative process is a mistake because administrative processes limit the number of aspects of the process approach that can be practiced by an auditor-in-training. Administrative processes rarely have equipment that requires validation or calibration, and both the process inputs and outputs consist only of paperwork, forms, or computer records. With raw materials and finished goods to process, the job of the auditor is more challenging, because there is more to be aware of.

4. Not providing honest feedback is a huge mistake. Auditors need to be thick-skinned, or they don’t belong in a role where they are going to criticize others. Before you begin telling other people how to improve, you first need to self-reflect and identify your own strengths and weaknesses. Understanding your perspective, strengths, weaknesses, and prejudices is critical to being an effective assessor. As a trainer, it is your job to help new auditors to self-reflect and accurately rate their performance against objective standards.

5. “Silent Shadowing” has no value at all. By this, I mean shadowing another auditor without asking questions. If you are a trainee, you should be mentally pretending you are doing the audit. Whenever the trainer does something different from the way you would do things, you should make a note so you can ask, “Why did you do that?” If you are the trainer, you should also be mentally pretending you are doing the audit. It is not enough to be present. Your job is to identify opportunities for the trainee to improve. The better the trainee, the more challenging it becomes to identify areas for improvement.  This is why training other auditors have helped me improve my auditing skills.

Auditor shadowing in second-party audits

If you are developing a new supplier quality engineer that is responsible for performing supplier audits, it is recommended to observe the auditor during some actual supplier audits. Supplier audits are defined as second-party audits in the ISO 19011 Standard. The purpose of these audits is not to verify conformity to all the aspects of ISO 13485. Instead, the primary purpose of these audits is to verify that the supplier has adequate controls in place to consistently manufacture conforming products for your company. Therefore, processes such as Management Review (Clause 5.6) and Internal Auditing (Clause 8.2.2) are not typically sampled during a second-party audit.

The two most valuable processes for a second-party auditor to sample are 1) incoming inspection, and 2) production controls. Using the process approach to auditing, the second-party auditor will have an opportunity to verify that the supplier has adequate controls for documents and records for both of these processes. Training records for personnel performing these activities can be sampled. The adequacy of raw material storage can be evaluated by following the flow of accepted raw materials, leaving the incoming inspection area. Calibration records can be sampled by gathering equipment numbers from calibrated equipment in use by both processes. Even process validation procedures can be assessed by comparing the actual process parameters being used in manufacturing with the documented process parameters in the most recent validation or re-validation reports.

My recommendation is to have the trainee shadow the trainer during the process audit of the incoming inspection process, and for the trainer to shadow the trainee during the process audit of production processes. In between the two process audits, the trainee should be asking questions to help them fully understand the process approach to auditing. Supplier auditors should also be coached on techniques for overcoming resistance to observing processes that may involve trade secrets, or where competitor products may also be present. During the audit of production processes, the trainer may periodically prompt the trainee to gather the information that will be needed for following audit trails to calibration records, document control, or for comparison with the validated process parameters. The “teachable moment” is immediately after the trainee missed an opportunity, but while the trainee is still close enough to go back and capture the missing details.

Auditor shadowing in third-party audits

Use your FDA inspections and ISO certification audits as an opportunity to shadow experienced auditors and to learn what they are looking for.

If you are going to shadow a third-party auditor, I recommend two specific people to “shadow” the auditor. First, the process owner should be the guide for whichever process is being audited. This is the person that will be responsible for addressing any nonconformities found in the area, and they should be present during interviews–although they should be coached on when to comment and when to remain quiet and simply observe.  Second, the person that performed an internal audit of the process being audited should be present if at all possible. This person will benefit from seeing how a professional third-party auditor performs a process audit because they will know which things to look for in the future so that auditees in that area are prepared for the next external audit.

For other sources of information related to auditor shadowing, please check out the following links:

1. Internal Auditor Training – Shadowing external auditor? – from Elsmar Cove

2. Developing Supplier Quality Auditor Training Programs – by Seth Mailhot at Nixon Peabody