Process Approach to Auditing

The process approach to auditing is demonstrated using Turtle Diagrams as a tool instead of using traditional auditor checklists.

ISO 9001 Quality System Principles

ISO 9001 is the general quality system standard that was created in 1994. The ISO 9001 standard forms the basis for all other international quality system standards–including ISO 13485. There are seven quality system principles that form the basis of ISO 9001:

1. 1. Customer Focus
   2. Leadership
   3. Engagement of People
   4. Process Approach
   5. Improvement
   6. Evidence-based Decision Making
   7. Relationship Management

Is there more than one method of auditing?

There are several different approaches to conducting an audit:

1. Regulatory checklist
2. Procedural approach
3. Element approach
4. Contract audit
5. Product audit
6. Process approach

Each of these approaches to auditing is a valid approach. However, each approach has benefits and disadvantages. Therefore, an audit program manager should be knowledgeable of each approach when they are making recommendations to top management with regard to the audit program schedule.

Regulatory Checklist

The most common method of auditing is to use a regulatory checklist. This is the approach used by certification bodies for the Medical Device Single Audit Program (MDSAP). For each regulatory requirement or standard, there is a row in a checklist. This approach is also known as the element approach, because each clause or section of the applicable requirement constitutes an “element.” The requirements are in the left column, and the requirement is usually referenced (e.g., clause number). The subsequent columns of the checklist are intended to document which documents and records the auditor reviewed. The last column of the checklist is where the auditor documents what they looked for in those documents and records.

Each audit checklist is based on a standard or regulation. Therefore, if there are multiple applicable standards and regulations, multiple checklists would be needed to use this approach exclusively. The biggest disadvantage of this approach is that auditors use the checklist as a crutch and will ask only the questions on the checklist. The greatest benefit of this approach is that auditors can verify that all the requirements of a standard or regulation have been met. This is generally the best approach for internal auditing just prior to an initial certification audit (i.e., Stage 1 and Stage 2).

Procedural approach to auditing

The procedural approach to auditing is similar to the element approach. However, a checklist does not need to be created in advance, and for supplier audits, it is not practical to invest the time in creating a checklist for a supplier’s procedures. In the procedural approach, the auditor reviews the procedure and identifies important elements of the procedure to verify are being performed. Often, this is achieved by making a copy of the procedure and highlighting requirements in the procedure to verify.

A contract audit is also similar to a procedural audit, but instead of using a procedure as the basis for the requirements, a supplier contract is used instead. If the supplier contract includes a quality agreement with all of the quality system and regulatory requirements defined, this approach may duplicate all requirements of a regulatory checklist. The biggest disadvantage of this approach is that it is unable to identify failures in the interactions between processes. This approach is ideal as an audit of a new or revised procedure, but the auditor may need to supplement this approach with the process approach to identify gaps in those interactions.

What is a product audit?

Product auditing involves auditing everything associated with a single product or product family. This is typically done when a new product is being launched, and the medical device manufacturer wants to audit manufacturing processes prior to launch (or a supplier if the manufacturing is outsourced). The auditor may review anything in the device master record (DMR – 21 CFR 820.181 in FDA QSR) or medical device file (MDF – ISO 13485:2016, Clause 4.2.3).

Product audits are also the approach used for unannounced audits. Unannounced auditors verify that the devices being manufactured and inspected match the drawings and specifications in the technical documentation that is approved for CE Marking. This verification includes inspection and testing methods for product release. Certification body auditors and FDA inspectors are both trained to focus on design changes, inspection methods, and especially the final test of devices prior to release. This focus is a risk-based approach where auditors sample the most important processes. If you are conducting a product audit, we recommend mirroring this approach.

What is the process approach to auditing?

The process approach is just a different way of organizing audits. Instead of auditing by clause, procedure, or product, you audit each process. Typical processes include:

Why the Process Approach is Recommended

The process approach to auditing is preferred over all other methods for two reasons. First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments, the process approach will expose it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked.

Second, the process approach is a more efficient way to cover all the clauses of a standard than auditing each clause individually (i.e., the element approach). My rationale for the claim of greater efficiency is simple. There are 34 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are incorporated into each process audit.

For example, each process audit requires a review of both the records as input and the outputs. In a process audit, training records can be sampled for each employee interviewed during the audit as part of an audit trail. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment. The tool we use to teach the process approach is the “Turtle Diagram.” The diagram below illustrates the origin of the name.

Interviewing with the Process Approach

The first skill to teach a new auditor is how to interview. Each process approach audit should begin with interviewing the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next, most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique because it is “closed-ended” or yes/no. Closed-ended questions do little to gather objective evidence. Instead, start your interview with this simple request: “Please describe the process?” A process description gives you a general overview of the process if you are unfamiliar with it.

After receiving a general overview, try asking this question: “How do you know how to start the process?” Inspectors know that there is material for incoming inspection because raw materials are in the quarantine area. Companies use visual systems, electronic materials requisition and planning (MRP) systems, and paper-based systems to notify QC inspectors that the product is ready to be inspected. As an auditor, you are looking for a record to trigger the inspection process. A follow-up question is, “What are the outputs of the inspection process?” Once again, auditors need documents and records to review. Sampling inspection records and any associated records (e.g., certificates of analysis) are records the auditor samples to verify the effectiveness of the inspection process (i.e., Clause 7.4.3) and the process for control of records (i.e.,  Clause 4.2.4). The process approach allows the auditor to verify compliance with two clauses simultaneously.

The next step of the process approach is to “determine what resources are used by incoming inspection.” This includes gauges used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list, the auditor should select people to interview and follow up with a request for training records.

The sixth step is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures, this could potentially waste valuable time. Instead, you can ask the inspector to show you where to find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

Challenging Process Owners

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should review department quality objectives and assess if the department objectives are linked with company quality objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step is also where the auditor can sample and review the monitoring and measurement of processes, and the trend analysis can be verified to be input into the CAPA process.

In my brief description of the process approach, I used the incoming inspection process. I typically choose this process for training new auditors because it is a process that is quite similar in almost every company, and it is easy to understand. More importantly, however, the incoming inspection process does an effective job of covering more clauses of the Standard than most audits. Therefore, new auditors get an appreciation for how almost all the clauses can be addressed in one process audit. If you are interested in learning more about Turtle Diagrams and the process approach to auditing, please register for our webinar on the process approach to auditing.