Filter posts by category

ISO Auditing

Internal Auditing, Supplier Auditing, Lead Auditor, Internal Auditor, ISO 19011, and ISO 17021.

Audit Findings – How to communicate good and bad findings.

4 Comments / ISO Auditing / By / , ,

This blog describes best practices for communicating audit findings during an audit, in the closing meeting, and in the audit report.

Would you like to be surprised by an auditor with a major nonconformity? Of course not! Nobody likes that kind of surprise. However, do you know how to effectively communicate your audit findings during the audit, in the closing meeting, and in your audit report?

Audit findings should be communicated at the time the objective evidence is gathered, and it should be clearly stated whether you think the finding is a nonconformity or an opportunity for improvement. Give the auditee an opportunity to correct you.

Audit Finding Example

If you are auditing the process for creating a medical device file, and you are unable to find evidence of product specifications (i.e., ISO 13485:2016, Clause 4.2.3b), then you should restate the requirement and explain why this is a nonconformity. It may be a nonconformity because that requirement is not included in the procedure or index for your medical device file. It may be a nonconformity because the product specification is obsolete and needs to be updated. It may be a nonconformity because you were unable to find the product specification anywhere in the device master record (DMR) index or technical file index. You might also be surprised to learn that product specifications are included in the product user manual, but the process owner forgot that because they were very nervous. The morning after the audit, the process owner may be prepared to show you exactly what you were looking for, including procedural requirements and training.

How do you respond when findings are resolved

Some auditors are irritated when they spend time following the audit trail, and after they have taken the time to write a nonconformity, the auditee finally produces the evidence requested. Some auditors say, “It’s too late. You were unable to provide the record when it was requested.” That’s not a value-added finding. The right approach is to say, “Excellent! Now we don’t need to issue a nonconformity or investigate the root cause for a missing product specification.” You might also add, “As a follow-up to this audit, consider ways you can make the product specifications and other required technical documentation easier to find during an audit.” If a similar scenario is repeated during the audit, you might consider writing an OFI beginning with the word “Consider.” However, be careful of suggesting solutions. Medical Device Academy adds cross-references to requirements in each procedure, but that is time-consuming and not required.

How to grade an audit finding

In our example above, if evidence of the product specification was not found, that would be a nonconformity. If several other requirements in the medical device file were not available, it would still be a nonconformity. Some people would grade a single lapse as a “minor,” but if multiple requirements are missing they would grade the finding as a “major.” This is not enough to deserve the grading of a “major” but grading subjectivity is difficult to avoid. The specification might exist, but it was accidentally omitted from the file. The specification might not be documented for the file sampled, but it may be easily identified for other product files. The specification might only be missing, because a new employee forgot it and the file was not thoroughly reviewed yet. Therefore, the auditor should consider the missing element an “audit trail.” They should review previous audit reports for similar nonconformities, sample additional requirements, sample other files, and review training records before determining the grading.

Why do the GHTF and MDSAP guidance documents use quantitative grades?

 In 2012, the Global Harmonization Task Force (GHTF) published a guidance document for grading auditing findings. That guidance proposed a quantitative scoring system with a range of 1-5. Initially, I thought this system was overly complicated. Later, the Medical Device Single Audit Program (MDSAP) adopted the same quantitative scoring system. Since many of our clients adopted MDSAP, we had to learn the MDSAP audit approach and we had to learn how to grade audit findings quantitatively. After using the new system, I realized that the quantitative approach was faster because the objective grading reduced the time required to make a decision on the grade of the finding.

Direct and indirect impact on product safety and performance

Experienced auditors have most of ISO 13485 memorized, and they usually know which requirements are included in Clauses 4.1-6.3, and which requirements are found later in the standard. Therefore, identifying whether the finding is “direct” or “indirect” is easy. Clauses 4.1-6.3 are indirect clauses, with the exception of 4.2.3 which is direct. There is also one exception to the direct clauses; Clause 8.2.4 is the only clause within Clauses 6.4-8.5.3 that is indirect. It would be easy to persuade someone that there should be additional exceptions, but it would just make the process slower and subjective. Using the clause number for each requirement to determine the initial scoring makes the process faster and more reliable.

When do escalation rules apply?

There are three escalation rules to consider when grading a nonconformity in the GHTF or MDSAP audit approach. The image below is included in our CAPA form to help remind people of the scoring. The first rule is specific to a repeat nonconformity in the past three (3) years. The second escalation rule is controversial because many people believe the absence of a procedure or records should be sufficient by itself to escalate a finding. However, it’s just a grade, and if the finding is escalated, we want there to be no doubt that the process is not able to meet the requirements. The final escalation rule is the most serious because shipping nonconforming products requires implementation of a recall or field service corrective action (FSCA). Medical Device Academy applies these same three escalation rules when deciding whether a finding is a “major” if a client does not use the MDSAP audit scoring system. This ensures that our grading is objective and it is based on international guidance. We use this same scoring system for internal auditing, supplier auditing, and CAPAs.

Scoring of NCs 1024x254 Audit Findings How to communicate good and bad findings.

Audit findings must include more than nonconformities

In the paragraphs above, we discussed the grading of nonconformities; however, reporting audit findings involves more than just grading nonconformities. ISO 19011:2011 is the official guidance document for auditors of Quality Management Systems, and ISO 13485 is the quality system standard for medical device manufacturers. Section 6.4.2 of this Standard explains best practices for an opening meeting.

  1. Method of reporting audit findings, including grading, if any
  2. Conditions under which the audit may be terminated
  3. Time and place of the closing meeting
  4. How to deal with possible findings during the audit
  5. System for feedback from the auditee on findings or conclusions of the audit
  6. Process for complaints and appeals

The opening meeting is the ideal opportunity to outline how you and your team will present audit findings and to clarify that you will discuss both the strengths and weaknesses of the quality system verbally in the closing meeting and in the audit report. If the auditee is new to auditing, you might even explain the three-part structure of how nonconformities are written.

Conditions for Termination

The option to terminate an audit is typically reserved for a certification audit where multiple major nonconformities are identified, and there is no point in continuing. Termination is highly discouraged because it is better to be aware of all minor and major nonconformities immediately, rather than waiting until the certification audit is rescheduled. The certification body will charge you for their time anyway.

Another reason for termination is when an auditor acts unreasonably or inappropriately. This is rare, but it happens. If the audit is terminated, you should communicate this to upper management at both the certification body and the company, regardless of which side of the table you sit on. For FDA inspections, this is not an option. For audits performed by Notified Bodies, there is the possibility of suspension of a certificate in response to audit termination. Therefore, I always recommend appealing after the fact, instead of termination. Appealing also works for FDA inspections.

Closing Meeting

The closing meeting should be conducted as scheduled, and the time/location should be communicated to upper management in the audit agenda and during the opening meeting. Top management won’t be happy about nonconformities, but failure to communicate when the closing meeting will be conducted will irritate them further. You should also ensure that a teleconference invitation is set up in advance for the closing meeting, allowing top management to participate remotely if necessary.

At the closing meeting, the auditee should never be taken by surprise. If an issue remains unfulfilled at the closing meeting, the auditee should expect a minor nonconformity—unless the issue warrants a major nonconformity. Since a minor nonconformity can result from a single lapse in fulfilling a requirement, it is challenging for an auditee to argue that an issue does not warrant a minor nonconformity. Typically, the argument is that you are not consistent with other auditors. The most common response to that issue is, “Audits are just a sample, and previous auditors may not have seen the same objective evidence.” The more likely scenario, however, is that the previous auditor interprets the requirements, rather than reviewing them with the client and ensuring both parties agree before a finding is issued.

If a finding is major, the auditee should have very few questions. Additionally, I often find that the reason for a major nonconformity is a lack of management commitment to address the root cause of the problem. Issuing a major nonconformity is sometimes necessary to get management’s attention.

Regardless of the grading, all audit findings will require a corrective action plan—even an FDA warning letter requires a CAPA plan. Therefore, a major nonconformity is not a disaster. You just need to create a more urgent plan for action.

How to deal with audit findings

All guides and auditees should be informed of potential findings at the time an issue is identified. This is important so that an auditee has the opportunity to clarify the evidence being presented. Often, nonconformities result from miscommunication between the auditor and the auditee. This often occurs when the auditor lacks a thorough understanding of the process being audited. It is a tremendous waste of time for both sides when this occurs. If there is an actual nonconformity, it is also important to gather as much objective evidence as possible for the auditor to write a thorough finding and for the auditee to prepare an appropriate corrective action plan in response to the discovery.

Feedback from the Auditee

As an auditor, I encourage auditees to provide honest feedback directly to me and to management, so that I can continue to improve. If you are providing feedback about an internal auditor or a supplier auditor, you should always give feedback directly to the person before going to their superior. You are both likely to work together in the future, and you should give the person every opportunity to hear the feedback firsthand.

When providing feedback from a third-party certification audit, you should know that there will be no negative repercussions against your company if you complain directly to the certification body. At most, the certification body will assign a new auditor for future audits and investigate the need for taking action against the auditor. In all likelihood, any action taken will be “retraining.” I never fired somebody for a single incident—unless they broke the law or did something unsafe. The key to providing feedback, however, is to be objective. Give specific examples in your complaint, and avoid personal feelings and opinions.

Complaints and appeals of audit findings

As an auditor, one of the most important (and difficult) things to learn is how to issue a nonconformity—especially a major. This is typically done at the closing meeting of an audit; however, the closing meeting is not where the process of issuing the nonconformity begins. Issuing a nonconformity starts in the opening meeting.

As the auditee, you should ask for the contact information of the certification body during the opening meeting. Ask with a smile—just in case you disagree, and so you can provide feedback (which might be positive). As the auditor, you should always provide the certification body’s contact information (if they are a third-party auditor). If you are conducting a supplier audit or an internal audit, you probably know the auditor’s boss, and there is perhaps no formal complaint or appeals process. In the case of a supplier audit, the customer is always right—even when they are wrong.

Additional Auditor Training

If you would like to learn more about auditing methods and best practices, consider registering for our Lead Auditor Training Course.

Audit Findings – How to communicate good and bad findings. Read More »

Auditing Design Controls

4 Comments / ISO Auditing / By / , ,

Learn how to apply the process approach when auditing design controls and discover how audit checklists fail to identify problems.

Examples of auditing design controls with an audit checklist:

Audit checklists list each requirement in a standard or regulation. When auditing design controls, an audit checklist lists each of the ten subsections of the ISO 13485:2016 standard. For example, here are potential audit checklist questions for the first three subsections:

  • Clause 7.3.1 – Which procedure(s) defines your design control process?
  • Clause 7.3.2 – What is the design plan for your new product? When was the plan last updated?
    • a) What are the phases of your project plan?
    • b) In which phases are design reviews planned?
    • c) In which project phases are verification, validation, and design transfer activities performed?
    • d) Who is responsible for approval of design changes during the project? Who is responsible for updating the design plan as the project progresses?
    • e) How do you demonstrate traceability between hazards, design inputs, design outputs, and testing requirements?
    • f) What equipment and software do you use? What are the training requirements for your design team?
  • Clause 7.3.3 – How do you document design inputs? When were the design inputs reviewed and approved?
    • a) What are the performance requirements? What are the safety requirements? What are the usability requirements?
    • b) What are the applicable regulatory requirements and standards?
    • c) Which hazards have you identified?
    • d) Which design requirements were transferred from a previous design of your device?
    • e) How do you ensure that essential principles of safety and performance are met?

How can you improve the first question on the checklist?

The problem with the above questions is that they do not evaluate linkages with other processes. For example, when you ask what procedure(s) defines the design control process (e.g., SYS-008, Design Control Procedure and SYS-006, Change Control Procedure), you can also ask the revision of that procedure(s). The record associated with that document change can be used to evaluate the effectiveness of the document control process. For example, were all the job functions that reviewed and approved the previous version of the procedure represented during the review and approval of the current revision? The procedure itself can also be reviewed to make sure that it includes the appropriate elements for a procedure (e.g., scope, references, roles/responsibilities, description of the process, revision history, etc.). This approach to verifying the effectiveness of the document control process can be applied to every procedure within every process.

What are the problems with using an audit checklist?

An audit checklist is always based on the same regulation or standard. Therefore, when establishing a quality system, using an audit checklist is ideal preparation for the initial certification audit. However, if internal auditors ask the same questions during every audit, then auditees begin to anticipate the questions. Anything not included in the audit checklist may be overlooked. For example, when auditing design controls, there are no requirements for supplier controls mentioned in Clause 7.3. Therefore, an audit checklist will not include any questions about the qualification of suppliers that support design and development (e.g., software developers), as those requirements are already addressed in Clause 7.4.2 (i.e., purchasing information).

Another problem with using audit checklists is that auditors may rely too heavily on the checklist as a crutch. Auditors are supposed to plan the audit agenda based on the importance of areas and the results of previous audits. If the auditor relies solely on the checklist, each clause is assigned equal importance — regardless of its importance or the results of previous audits. Auditors also need to verify compliance with all applicable standards. Creating a checklist for risk management (i.e., ISO 14971), software lifecycle management (i.e., IEC 62304), usability engineering (i.e., IEC 62366), and information technology security management (i.e., ISO 27001) would be extremely time-consuming, and auditors would not be able to complete all of the checklist questions. Auditors require a more efficient method to assess the effectiveness of a process and verify compliance with requirements.

Basics of the process approach to auditing

The process approach to auditing is different. Instead of creating a checklist that is specific to the requirements for each process in the standard, the process approach relies on asking seven basic questions and then following the audit trails presented by the answers to those questions. The tool we use to help us remember the seven questions is a “turtle diagram.” The shape of the turtle has seven elements:

  1. body (description of the processes)
  2. head (inputs to the process)
  3. tail (outputs from the process)
  4. leg #1 (what equipment and software is required)
  5. leg #2 (who performs the process)
  6. leg #3 (what procedures and forms are used in the process)
  7. leg #4 (which metrics are used to monitor the process)

Note: It’s only 7 steps. You don’t have to tattoo a turtle diagram on your arm.

The diagram below uses the image of a turtle to remind you of the seven elements, but we added the subclauses from ISO 13485 that are related to auditing design controls. A brief summary of how these subclauses are related is provided in the video above; more details on each part of the turtle diagram are provided below, specifically for design controls.

Auditing Design Controls 9 30 2025 1024x576 Auditing Design Controls

Step 1 – Describe the process

The first step in creating a process audit is to identify the process owner and conduct an interview with them. We recommend doing this in their office, not in the conference room, for three reasons:

  1. Auditor effectiveness will improve if you periodically get up and walk around, because it will make you more alert.
  2. Conference rooms isolate auditors from daily operations, and the auditor may not gain an appreciation for where people perform their work or the proximity of the design team leader to the rest of the team.
  3. Auditees will be more relaxed in their office when being interviewed than they would be in a conference room.

After the process owner provides a brief description of the process, try to get answers to steps 2-7 directly from them in the same interview. Asking open-ended questions to prevent “yes/no” responses will be helpful. You also need a comprehensive understanding of the design control process before interviewing other team members or requesting design records.

Step 2 – Inputs

Even when auditors use the process approach to auditing, this part of the turtle diagram is frequently incomplete when auditing design controls. The obvious answer is to review the auditee’s approval of design inputs. This is a required record for design controls in Clause 7.3.3; however, it is not the only process input for design controls. As stated in Clause 7.3.3, “These inputs shall include…c) applicable output(s) of risk management.” Additionally, Clause 8.2.1 states, “The information gathered in the feedback process shall serve as potential input into risk management for monitoring and maintaining the product requirements as well as the product realization or improvement processes.” Therefore, both risk management and post-market feedback should be included as inputs to the design process. Using the process approach when auditing design controls will show you if the interactions between the risk management process, post-market surveillance process, and the design control process are adequate. Other inputs that should be considered for the design control process include regulatory requirements, such as:

  • Common Specifications (EU)
  • General Safety & Performance Requirements
  • Applicable Safety and Performance Standards
  • Applicable FDA Guidance

Step 3 – Outputs

Most auditors do an excellent job of covering the process outputs when auditing design controls (or any process), as the outputs typically include records, and auditors document which records they reviewed in their audit report. For the design controls process, the Design History File (DHF) (i.e., Clause 7.3.10) is the primary record sampled, and the Device Master Record (DMR) is the second most commonly sampled record. With the changes to the FDA requirements for the QMSR, auditors will be looking for a Medical Device File (i.e., Clause 4.2.3) instead; however, the records should remain the same, with just a new name. If the device is CE marked, there should also be a technical file or a technical file index.

Step 4 – What Resources

A critical part of auditing is to verify that a process is not only documented but also implemented. To implement any process, equipment, or software will likely be necessary. For the implementation of design controls, most companies utilize quality system software to manage documents and records for each design project. For example, Grand Avenue Software could be used for managing the medical device file (i.e., Clause 4.2.3), and AdaptivRisk may be used for managing the risk management file. There may also be some calibrated testing equipment that requires validation, calibration, and maintenance. Therefore, this step in the turtle diagram usually involves the following ISO 13485 clauses:

  • Clause 7.5.6 – process validation
  • Clause 6.3 – infrastructure (i.e., maintenance)
  • Clause 7.6 – monitoring of measurement equipment (i.e., calibration)

This is typically the step of a process audit where the auditor needs to identify “what resources” are used in the process. However, only companies that have software systems for design controls have resources dedicated to Design and Development. I have indicated this in the “Turtle Diagram” presented above.

Step 5 – Who

The next step in the process approach to auditing design controls is to identify who is assigned to the design team for a design project. Sometimes companies assign large teams. In this case, the auditor should focus on the team members who must review and approve design inputs (see Clause 7.3.2) and design outputs (see Clause 7.3.4). All team members should have training records (i.e., Clause 6.2) for Design Control procedures and Risk Management procedures. However, if the device includes software and internet connectivity, some members of the design team will require additional training on specific standards and protocols. It is also necessary to outsource processes that cannot be performed by the manufacturer, such as software development, cybersecurity testing, biocompatibility testing, and EMC testing. For these outsourced processes, the company must document the supplier’s qualification and establish a written agreement with that supplier (i.e., Clause 7.4.2). Examples of agreements could be a supplier quality agreement, a consulting contract, or a signed GLP testing protocol.

Step 6 – Standard Operating Procedures (SOPs) or “How done”

Auditors using the process approach to auditing often discover ineffective processes when they expand the scope of design controls beyond the scope of the design control procedure. The design team leader will identify the design control procedure(s) and forms. However, the auditor should also request copies of the risk management procedure and other related procedures. The other procedures may have different process owners, and the design team leader may not be adequately trained in those procedures. The auditor should not read and review these procedures. Auditors never have the time to do this. Instead, ask the process owner to identify specific procedures or clauses within procedures where clauses in the ISO Standard are addressed. If the process owner knows exactly where to find what you are looking for, they’re training was effective, or they may have written the procedure(s). If the process owner has trouble locating the clauses you are requesting, spend more time sampling training records. You may also want to ask if there is another person who is more familiar with the procedure. This step of the process approach is also when you should be sampling records of document control (i.e., Clause 4.2.4).

Step 7 – Metrics

The seventh step of the turtle diagram is typically where the auditor discovers the most value-added findings. The auditor will ask the process owner to identify some metrics (i.e., Clause 8.2.5) or quality objectives (i.e., Clause 5.4.1) they are using to monitor and improve the design and development process. This is a struggle for many process owners — not just the design team leader. If any metrics are not performing up to expectations, there should be evidence of actions being taken to address this. If the process owner is not tracking metrics, you may want to review how closely the actual project schedule aligns with the design project plan. Design projects are frequently delayed because the design team either does not request quotes early enough or does not involve the supply chain manager soon enough, or both. There is also considerable benefit derived from conducting retrospective reviews at the end of design phases and at the project’s conclusion. The team will identify changes in time estimates that should be considered for future design projects or other ongoing projects.

Supplementary questions for auditing design controls

After all seven steps of the turtle diagram are complete, the process audit is not yet complete. The auditor needs to sample records and follow audit trails to ensure thoroughness. Therefore, additional records need to be sampled. We recommend sampling design changes because this is where inspectors and third-party auditors will typically focus. These external auditors will be looking for design changes that need regulatory approval and may not have been submitted for market authorization. The auditor may also sample using a risk-based approach when sampling design changes. In particular, we recommend looking for the following types of changes: 1) vendor change, 2) specification change, and 3) process change. By doing this, the audit will also cover the following clauses in ISO 13485:2016: 7.4 (purchasing), 7.3.9 (design changes), 7.5.6 (process validation), 7.1 (risk management), and 4.2.5 (control of records). If you would like to learn more about design changes, please watch our Design Changes Webinar.

Record sampling for auditing design controls

FDA inspectors and third-party auditors have similar approaches to auditing design controls. Both will begin by reviewing your procedure to verify that it includes all of the required elements of ISO 13485:2016, Clause 7.3. Next, they will sample a recent design project that was completed and request a copy of the design history file (DHF). Many design projects are behind schedule, and therefore, this is an important metric for most companies. Now that you have completed your “Turtle Diagram,” if you have more time, you can conduct interviews with team members to review their roles in the design process. You could also sample-specific Technical Files, as I indicated above. If you are performing a thorough internal audit, I recommend doing both. To learn more about using the process approach to auditing, you can register for our webinar on the topic.

Auditing Design Controls Read More »

Process Approach to Auditing

4 Comments / ISO Auditing / By / , ,

The process approach to auditing is demonstrated using Turtle Diagrams as a tool instead of using traditional auditor checklists.

ISO 9001 Quality System Principles

ISO 9001 is the general quality system standard that was created in 1994. The ISO 9001 standard forms the basis for all other international quality system standards–including ISO 13485. There are seven quality system principles that form the basis of ISO 9001:

    1. Customer Focus
    2. Leadership
    3. Engagement of People
    4. Process Approach
    5. Improvement
    6. Evidence-based Decision Making
    7. Relationship Management

Is there more than one method of auditing?

There are several different approaches to conducting an audit:

  1. Regulatory checklist
  2. Procedural approach
  3. Element approach
  4. Contract audit
  5. Product audit
  6. Process approach

Each of these approaches to auditing is a valid approach. However, each approach has benefits and disadvantages. Therefore, an audit program manager should be knowledgeable of each approach when they are making recommendations to top management with regard to the audit program schedule.

Regulatory Checklist

The most common method of auditing is to use a regulatory checklist. This is the approach used by certification bodies for the Medical Device Single Audit Program (MDSAP). For each regulatory requirement or standard, there is a row in a checklist. This approach is also known as the element approach, because each clause or section of the applicable requirement constitutes an “element.” The requirements are in the left column, and the requirement is usually referenced (e.g., clause number). The subsequent columns of the checklist are intended to document which documents and records the auditor reviewed. The last column of the checklist is where the auditor documents what they looked for in those documents and records.

Each audit checklist is based on a standard or regulation. Therefore, if there are multiple applicable standards and regulations, multiple checklists would be needed to use this approach exclusively. The biggest disadvantage of this approach is that auditors use the checklist as a crutch and will ask only the questions on the checklist. The greatest benefit of this approach is that auditors can verify that all the requirements of a standard or regulation have been met. This is generally the best approach for internal auditing just prior to an initial certification audit (i.e., Stage 1 and Stage 2).

Procedural approach to auditing

The procedural approach to auditing is similar to the element approach. However, a checklist does not need to be created in advance, and for supplier audits, it is not practical to invest the time in creating a checklist for a supplier’s procedures. In the procedural approach, the auditor reviews the procedure and identifies important elements of the procedure to verify are being performed. Often, this is achieved by making a copy of the procedure and highlighting requirements in the procedure to verify.

A contract audit is also similar to a procedural audit, but instead of using a procedure as the basis for the requirements, a supplier contract is used instead. If the supplier contract includes a quality agreement with all of the quality system and regulatory requirements defined, this approach may duplicate all requirements of a regulatory checklist. The biggest disadvantage of this approach is that it is unable to identify failures in the interactions between processes. This approach is ideal as an audit of a new or revised procedure, but the auditor may need to supplement this approach with the process approach to identify gaps in those interactions.

What is a product audit?

Product auditing involves auditing everything associated with a single product or product family. This is typically done when a new product is being launched, and the medical device manufacturer wants to audit manufacturing processes prior to launch (or a supplier if the manufacturing is outsourced). The auditor may review anything in the device master record (DMR – 21 CFR 820.181 in FDA QSR) or medical device file (MDF – ISO 13485:2016, Clause 4.2.3).

Product audits are also the approach used for unannounced audits. Unannounced auditors verify that the devices being manufactured and inspected match the drawings and specifications in the technical documentation that is approved for CE Marking. This verification includes inspection and testing methods for product release. Certification body auditors and FDA inspectors are both trained to focus on design changes, inspection methods, and especially the final test of devices prior to release. This focus is a risk-based approach where auditors sample the most important processes. If you are conducting a product audit, we recommend mirroring this approach.

What is the process approach to auditing?

The process approach is just a different way of organizing audits. Instead of auditing by clause, procedure, or product, you audit each process. Typical processes include:

  1. Design & development
  2. Purchasing
  3. Incoming inspection
  4. Assembly
  5. Final Inspection
  6. Packaging
  7. Sterilization
  8. Customer Service
  9. Shipping
  10. Management review
  11. CAPA
  12. Internal Auditing

Why the Process Approach is Recommended

The process approach to auditing is preferred over all other methods for two reasons. First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments, the process approach will expose it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked.

Second, the process approach is a more efficient way to cover all the clauses of a standard than auditing each clause individually (i.e., the element approach). My rationale for the claim of greater efficiency is simple. There are 34 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are incorporated into each process audit.

For example, each process audit requires a review of both the records as input and the outputs. In a process audit, training records can be sampled for each employee interviewed during the audit as part of an audit trail. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment. The tool we use to teach the process approach is the “Turtle Diagram.” The diagram below illustrates the origin of the name.

Turtle Diagram Process Approach to AuditingInterviewing with the Process Approach

The first skill to teach a new auditor is how to interview. Each process approach audit should begin with interviewing the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next, most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique because it is “closed-ended” or yes/no. Closed-ended questions do little to gather objective evidence. Instead, start your interview with this simple request: “Please describe the process?” A process description gives you a general overview of the process if you are unfamiliar with it.

After receiving a general overview, try asking this question: “How do you know how to start the process?” Inspectors know that there is material for incoming inspection because raw materials are in the quarantine area. Companies use visual systems, electronic materials requisition and planning (MRP) systems, and paper-based systems to notify QC inspectors that the product is ready to be inspected. As an auditor, you are looking for a record to trigger the inspection process. A follow-up question is, “What are the outputs of the inspection process?” Once again, auditors need documents and records to review. Sampling inspection records and any associated records (e.g., certificates of analysis) are records the auditor samples to verify the effectiveness of the inspection process (i.e., Clause 7.4.3) and the process for control of records (i.e.,  Clause 4.2.4). The process approach allows the auditor to verify compliance with two clauses simultaneously.

The next step of the process approach is to “determine what resources are used by incoming inspection.” This includes gauges used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list, the auditor should select people to interview and follow up with a request for training records.

The sixth step is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures, this could potentially waste valuable time. Instead, you can ask the inspector to show you where to find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

Challenging Process Owners

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should review department quality objectives and assess if the department objectives are linked with company quality objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step is also where the auditor can sample and review the monitoring and measurement of processes, and the trend analysis can be verified to be input into the CAPA process.

In my brief description of the process approach, I used the incoming inspection process. I typically choose this process for training new auditors because it is a process that is quite similar in almost every company, and it is easy to understand. More importantly, however, the incoming inspection process does an effective job of covering more clauses of the Standard than most audits. Therefore, new auditors get an appreciation for how almost all the clauses can be addressed in one process audit. If you are interested in learning more about Turtle Diagrams and the process approach to auditing, please register for our webinar on the process approach to auditing.

Process Approach to Auditing Read More »

Software vendors – How do you audit software developers?

1 Comment / ISO Auditing / By / ,

Learn how to qualify and audit software vendors to develop software as a medical device (SaMD) and software in a medical device (SiMD).

How do you audit medical device software developers?

Software medical devices are used to assist medical professionals. For example, radiologists use software to identify areas of interest for medical imaging. Do you know how to audit software vendors?

As a third-party auditor, I have had the pleasure of auditing software companies for CE Marking. When you audit a software company for the first time, this forces you to re-learn the entire ISO 13485 Standard. For example, if a company only produces software (i.e., software as a medical device or SaMD), there is very little to sample for incoming inspection and purchasing records. This is because the product is not physical—it’s software. Clauses of ISO 13485 related to sterility, implants, and servicing are also not applicable to SaMD products. If the software is web-based, the shipping and distribution clauses (i.e., – 7.5.1) might also present a challenge to an auditor.

The aspects of the ISO 13485 Standard that I found to be the most important to auditing software products were design controls and customer communication. Many auditors are trained in auditing the design and development of software, but very few auditors have experience auditing technical support call centers. When auditing a call center, most calls represent potential complaints related to software “bugs,” system incompatibilities with the operating system or hardware, and use errors resulting from the design of the user interface.

In most technical support call centers, the support person tries to find a workaround for identified problems. The problem with a “workaround” is that it is the opposite approach to the CAPA process. To meet ISO 13485 requirements, software companies must show evidence of monitoring and measuring these “bugs.” There must also be evidence of management identifying negative trends and implementing corrective actions when appropriate.

As an auditor, you should focus on how the company prioritizes “bugs” for corrective actions. Most software companies focus on the severity of software operations and the probability of occurrence. This is the wrong approach. Failure to operate is not the most severe result of medical device software failure. Medical device software can result in injury or death to patients. Therefore, it is critical to use a risk-based approach to the prioritization of CAPAs. This risk-based approach should focus on the severity of effects upon patients—not users. This focus on safety and performance is emphasized throughout the EU Medical Device Regulations and it is a risk management requirement in ISO 14971.

Referral to one of our favorite software developers

There are many vendors to choose from worldwide, but we prefer to work with smaller companies because our clients are start-up companies. We also prefer to work with vendors focused on the medical device industry. We also look for vendors that complement Medical Device Academy’s quality and regulatory expertise. Bold Type is a perfect example. The video below showcases the President and Founder–Jose Bohorquez. Bold Type provides software development services, cybersecurity consulting services, and software consulting services. If you are interested in speaking with Jose direct, please schedule a meeting with him online.

PS – We do not receive compensation from Bold Type–we just prefer to partner with firms that are ideal for our customers.

Software vendors – How do you audit software developers? Read More »

Hiring an Auditor

1 Comment / ISO Auditing / By /

In this article, you will learn how to hire an auditor to conduct medical device internal audits and supplier audits.

help wanted Hiring an Auditor
Stop begging people to help you audit. Learn how to recruit auditors more effectively.

Hiring and Auditor

Hiring an auditor, whether as a consultant or a permanent team member, is a critical decision that can drastically improve your quality management system and foster a culture of quality, or it can add no value and lead to disruption and frustration.  The purpose of this blog is to identify the qualities and training that make the best auditor to help you elevate your internal audit programs.  

Audit Program Structures 

Companies typically take one of the following approaches to address their internal audit requirements: 

  1. Train internal personnel with other primary functions as auditors and have them audit other departments.
  2. Hire an independent 3rd party to conduct the internal audits.
  3. Build an internal audit team that is independent of all other processes.  

Hiring an Auditor from Within 

Option 1 is common across the industry and is a personnel-efficient means of achieving the audit objectives. While this type of approach can sometimes be effective and may satisfy the basic requirement to conduct internal audits, there can be some drawbacks to this structure. Sometimes, when people were not hired specifically to be auditors and auditing is something they were asked to do in addition to their regular job, there is little to no motivation to develop auditing skills, and the audits lack a depth and thoroughness that ultimately reduces the value of the audit program. Proper internal recruiting and training of these auditors is crucial to ensuring audits are a useful value-added exercise and not a box-checking chore. 

To successfully recruit internal auditors serving in other roles, it’s important to motivate people to want to be an auditor. Let potential recruits know that employees with audit experience are more valuable to companies than those without. It exposes employees to upstream or downstream processes to better understand the overall operations and provides them the opportunity to make process improvements in both directions to their workflow. If you want to be effective and get promoted, you need to demonstrate value to your boss and top management. If you don’t understand what other departments need, how can you help them? No manager will promote a selfish, power-hungry hog. They promote team players that make others better. Auditing gives you the insight necessary to understand how you can do that.  

Once motivated and recruited, it’s important to ensure these employees have the skills and resources to be successful as auditors. To help develop their skills, training on audit processes and the responsibilities and role of an auditor in accordance with ISO 19011 will provide guidance on conducting audits and the basics of how to audit. Auditors should also be trained against the specific standard or regulation they are auditing against, which may include ISO 13485, 21 CFR 820, ISO 14971, EU MDR, and others. Resources that will support their activities may include process audit diagrams, checklists, examples of record requests, strategies for intelligent sample selection, and, of course, a clear definition of the regulatory and procedural requirements of the process that they are auditing.  

If you are looking for support in training your own employees to be internal auditors, we would be happy to outline or provide a training program specific to your company’s processes and products to ensure your auditors are competent and effective in their new role.  

Hiring a 3rd Party Auditor  

Option 2 can be useful to any company, but selecting the right auditor is essential to the success of this approach. The basic qualifications and qualities that I recommend companies look for when hiring an outside auditor are: 

  1. Experience – this includes industry experience and regulatory knowledge. An auditor with experience auditing or working for a company with similar devices, manufacturing processes, etc., will provide more value than an unfamiliar auditor. Regulatory knowledge and experience within your targeted markets are also important to evaluate to ensure that they are familiar with the standards and regulations against which they will be auditing.  
  1. Communication Skills – This is a make-or-break quality of auditors that can shift the substance of an audit from a value-added exercise to a disrupting and frustrating experience. You want to ensure that auditors are affable yet confident, able to communicate the usefulness of the audit for the purpose of process improvement and facilitate a productive dialogue, offering education and suggestions when issues or nonconformances arise.   
  1. Reputation and References – ask the auditor for references from previous clients. Contact the references to get feedback on their performance, reliability, and professionalism. This is a great way to evaluate an auditor’s communication skills and whether previous auditees gained value from the interaction.  
  1. Auditor Training – acceptable qualifications for an auditor can be defined by the company but may include lead auditor certification, demonstrated training on relevant standards with experience shadowing experienced auditors, and documented training on other relevant standards/regulations.  
  1. Audit Methodology – Inquire about how auditors plan, execute, and report on audits. What audit methodology does the auditor prefer for the scope of your audit, and why?  

There are many companies and consultants that offer 3rd party auditor services, but not all are created equal. Like the CAPA process, the internal audit program is a window into the culture surrounding quality that your company has, and by demonstrating that you are proactively policing yourself and seeing continuous improvement through an effective internal audit program will show regulators that your company has a commitment to quality.  

Hiring a Full-time Audit Team 

Option 3 is generally reserved for the resource-rich industry with operations that demand expansive continuous audit processes to justify the support of a full-time auditor or audit team. Hiring your own team benefits from the same considerations that come with hiring a 3rd party auditor; the ability for the auditors to become intimately familiar with the company, devices, and processes is valuable. For companies that do not have the need for full-time auditors, the same value of familiarity can come from building a trusted relationship with a third-party auditor or audit team, who can support your audit program year after year.  

Hiring an Auditor from Medical Device Academy  

Our goal at Medical Device Academy is to help you improve your quality system and provide valuable consulting advice to achieve improvements. We specialize in helping start-up companies achieve initial ISO 13485 certification, MDSAP certification, and CE Certification. Based on the scope of the audit and medical device, we will assign the most qualified team member. Some of our specific areas of expertise include auditing companies with manufacturing and machining, aseptic processing, agile software development, sterile products, medical device reprocessors, 3D printed manufacturing, and more. If you are interested in outsourcing any supplier or internal audit activities, you can check out our Audit Services page to get in touch or to learn more about our audit team.

Hiring an Auditor Read More »

Auditor shadowing as an effective auditor training technique

2 Comments / ISO Auditing / By /

This article reviews auditor shadowing as an effective auditor training technique, but we also identify five common auditor shadowing mistakes.

How do you evaluate auditor competency?

Somewhere in your procedure for quality audits, I’ll bet there is a section on auditor competency. Most companies require that the auditor has completed either a course for an internal auditor or a lead auditor course. If the course had an exam, you might even have evidence of training effectiveness. Demonstrating training competence is much more challenging. One way is to review internal audit reports, but writing reports is part of what an auditor does. How can you evaluate an auditor’s ability to interview people, take notes, follow audit trails, and manage their time? The most common solution is to require the auditor “shadow” a more experienced auditor several times, and then the trainer will “shadow” the trainee.

auditor with clip board 203x300 Auditor shadowing as an effective auditor training technique
If you are shadowing, you are taking notes, so you can discuss your observations with the person you are shadowing later. 

Auditor shadowing in 1st party audits

ISO 19011:2018 defines first-party audits as internal audits. When first-party auditors are being shadowed by a trainer or vice versa, there are many opportunities for training. The key to the successful training of auditors is to recognize teachable moments.

When the trainer is auditing, the trainer should look for opportunities to ask the trainee, “What should I do now?” or “What information do I need to record?” In these situations, the trainer asks the trainee what they should do BEFORE doing it. If the trainee is unsure, the trainer should immediately explain what, why, and how with real examples.

When the trainer is shadowing, the trainer should watch and wait for a missed opportunity to gather important information. In these situations, the trainer must resist guiding the trainee until after the trainee appears to be done. When it happens, sometimes the best tool is simply asking, “Are you sure you got all the information you came for?”

Here are five (5) mistakes that I observed trainers made when they were shadowing:

1. Splitting up, instead of staying together, is one of the more common mistakes I have observed. This happens when people are more interested in completing an audit rather than taking advantage of training opportunities. The trainee may be capable of auditing independently, but this is unfair to the trainee because they need feedback on their auditing technique. This is also unfair to the auditee because it is challenging to support multiple auditors simultaneously. When it is unplanned, trainers may not be available for both auditors. If an audit is running behind schedule, this is the perfect time to teach a trainee how to recover sometime in their schedule. Time management is, after all, one of the most challenging skills for auditors to master.

2. Staying in the conference room instead of going to where the work is done is a common criticism of auditors. If the information you need to audit can be found in a conference room, you could have completed the audit remotely. This type of audit only teaches new auditors how to take notes. These are necessary skills that auditors should master in a classroom before shadowing.

3. Choosing an administrative process is a mistake because administrative processes limit the number of aspects of the process approach that an auditor-in-training can practice. Administrative processes rarely have equipment that requires validation or calibration, and the process inputs and outputs consist only of paperwork, forms, or computer records. With raw materials and finished goods to process, the auditor’s job is more challenging because there is more to be aware of.

4. Not providing honest feedback is a huge mistake. Auditors need to be thick-skinned, or they don’t belong in a role where they will criticize others. Before you begin telling others how to improve, you must self-reflect and identify your strengths and weaknesses. Understanding your perspective, strengths, weaknesses, and prejudices is critical to being a practical assessor. As a trainer, it is your job to help new auditors to self-reflect and accurately rate their performance against objective standards.

5. “Silent Shadowing” has no value at all. By this, I mean shadowing another auditor without asking questions. You should mentally pretend you are doing the audit if you are a trainee. Whenever the trainer does something different from how you would do things, you should make a note to ask, “Why did you do that?” If you are the trainer, you should also mentally pretend you are doing the audit. It is not enough to be present. Your job is to identify opportunities for the trainee to improve. The better the trainee, the more challenging it becomes to identify areas for improvement.  This is why training other auditors have helped me improve my auditing skills.

Auditor shadowing in 2nd party audits

supply chain weakest link Auditor shadowing as an effective auditor training technique

Auditors responsible for supplier auditing are critical to supplier selection, supplier evaluation, re-evaluation, and the investigation of the root cause for any non-conformities related to a supplier. Auditor shadowing is a great tool to teach supplier auditors and other people responsible for supply-chain management what to look at and what to look for when they audit a supplier. If you are developing a new supplier quality engineer responsible for performing supplier audits, observing the auditor during some actual supplier audits is recommended. Supplier audits are defined as second-party audits in the ISO 19011 Standard. The purpose of these audits is not to verify conformity to all the aspects of ISO 13485. Instead, the primary purpose of these audits is to verify that the supplier has adequate controls to manufacture conforming products for your company consistently. Therefore, processes such as Management Review (Clause 5.6) and Internal Auditing (Clause 8.2.2) are not typically sampled during a second-party audit.

The two most valuable processes for a second-party auditor to sample are 1) incoming inspection and 2) production controls. Using the process approach to auditing, the second-party auditor will have an opportunity to verify that the supplier has adequate controls for documents and records for both of these processes. Training records for personnel performing these activities can be sampled. The adequacy of raw material storage can be evaluated by following the flow of accepted raw materials, leaving the incoming inspection area. Calibration records can be sampled by gathering equipment numbers from calibrated equipment used by both processes. Even process validation procedures can be assessed by comparing the actual process parameters being used in manufacturing with the documented process parameters in the most recent validation or re-validation reports.

I recommend having the trainee shadow the trainer during the process audit of the incoming inspection process and for the trainer to shadow the trainee during the process audit of production processes. The trainee should ask questions between the two process audits to help them fully understand the process approach to auditing. Supplier auditors should also be coached on techniques for overcoming resistance to observing processes involving trade secrets or where competitor products may also be present. During the audit of production processes, the trainer may periodically prompt the trainee to gather the information that will be needed for following audit trails to calibration records, document control, or for comparison with the validated process parameters. The “teachable moment” is immediately after the trainee misses an opportunity, but while the trainee is still close enough to go back and capture the missing details.

Are you allowed to shadow a 3rd party auditor or FDA inspector?

qsit inspection Auditor shadowing as an effective auditor training technique

Consider using 3rd party audits and inspections as an opportunity to shadow experienced auditors to learn what they are looking at and what they look for. In addition to shadowing an expert within your own company or an auditor/consultant you hire for an internal audit, you can also shadow a 3rd party auditor or an FDA inspector. This concept was the subject of a discussion thread I ran across on Elsmar Cove from 2005. The comments in the discussion thread supported the idea of shadowing a 3rd party auditor. The process owner (i.e., the manager responsible for that process) should be the guide for whichever process is being audited, and the process owner is responsible for addressing any non-conformities found in the area., The process owner should be present during interviews, but the process owner should refrain from commenting. The 3rd party auditor and the process owner need to know if the person being interviewed was effectively trained and if they can explain the process under the pressure of an audit or FDA inspection. If you are interested in implementing this idea, I recommend using one of two strategies (or both):

  1. Consider having the internal auditor that audited each process shadow the certification body auditor for the processes they audited during their internal audit. This approach will teach your internal auditor what they might have missed, and they will learn what the 3rd party auditors look for to simulate a 3rd party audit more effectively when conducting internal audits.
  2. Consider having the internal auditor that is assigned to conduct the next process audit of each process shadow the certification body auditor for that process. This approach will ensure that any nonconformities observed during the 3rd party audit are checked for the effectiveness of corrective actions during the next internal auditor. Your internal auditor will know precisely how the original nonconformity was identified and the context of the finding.

Auditor shadowing as an effective auditor training technique Read More »

Seven ways to improve quality auditor training

2 Comments / ISO Auditing / By / , ,

A five-day lead auditor course is never enough. Effective quality auditor training must include practical feedback from an expert.

What is required for quality auditor training?

The key to training auditors to audit is consistent follow-up over a long period of time (1-2 years, depending upon the frequency of audits). I recommend following the same training process that accredited auditors must complete. I have adapted that process and developed seven (7) specific recommendations.

Training the trainer

One of my clients asked me to create a training course on how to train operators. I could have taught the operators myself, but so many people needed training that we felt it would be more cost-effective to train the trainers. Usually, I have multiple presentations archived that I can draw upon, but this time I had nothing. I had never trained engineers on how to be trainers before—at least not formally. I thought about the problems other quality managers have had in training internal auditors and how I have helped the auditors improve. The one theme I recognized was that effective quality auditor training needs to include practical feedback from an experienced auditor. An expert auditor that is training new auditors needs to identify systematic ways to provide feedback, and setting a benchmark for the number of times feedback will be provided is really helpful.

Improve by observing yourself and other quality auditors

Observing someone else is a great way to learn when you are learning any new skill. Interns often do this, which is also a technique used to train new auditors. This technique is called shadowing. You can learn by watching, but eventually, you need to try to do tasks that are beyond your comfort level, and it is best to practice auditing with an expert watching you.

Practice team member audit preparation

Many of the internal auditing procedures we see require new auditors to conduct three audits as team members before they can audit independently. In contrast, notified body auditors join as team members for 10-20 audits before they can act as lead auditors. During the training period, auditors in training observe multiple lead auditors and multiple quality systems. Each audit allows auditors in training to write nonconformities and receive feedback from a lead auditor. At the beginning of quality auditor training, the focus must be on audit preparation. What are the areas of importance, what are the results of previous audits, are there any previous audit findings to close, etc? This preparation can even be done as practice for a hypothetical audit.

During quality auditor training, practice the opening and closing meetings

Opening and closing meetings are one of the first things to teach a new lead auditor. Have new lead auditors rehearse their first few opening and closing meetings with you in private before conducting the opening and closing meetings. Ensure the lead auditor has an opening/closing meeting checklist to help them. Recording practice sessions is enormously helpful because the trainee can watch and observe their mistakes. As trainees get more experience, the opening and closing meetings should have time limits. Finally, you might ask members of top management to challenge the lead auditor with questions. The lead auditor needs to be comfortable with their decisions and the grading of the audit findings.

How to practice audit team leadership

Have new lead auditors conduct team audits with another qualified lead auditor for 10-20 audits before you allow them to conduct an audit alone. Leading the opening and closing meetings is usually the first area new lead auditors master. The most complicated area to learn is managing a team of auditors. Team members will fall behind schedule during audits, or someone will forget to audit a process. As a lead auditor, you must complete the audits for your assigned processes and communicate with the entire team to ensure everyone else is on schedule. As an observer, you must let lead auditors make mistakes and help them realize them. Initially, a trainer will encourage new lead auditors to give themselves more than enough time. As their training progresses, the timing needs to be shorter and more challenging. Ultimately, you have to push the team beyond its capability to teach new lead auditors to recognize problem signs and teach them how to fix the problems.

Shadow auditors virtually with recordings

Live shadowing is challenging for experts and trainees because you are distracted by listening to the auditee and observing the auditor. However, if an audit is recorded, the person shadowing can watch the recording. The audit is already completed, and there is little need to concentrate on the auditee. A recording allows the observer to focus on the auditor. If a new auditor is conducting their first audit, an expert should shadow the trainee for 100% of the audit. Gradually the observation can decrease with each subsequent audit.

Practice note-taking with recorded audits

Taking detailed notes is something that experts take for granted, but I learned a lot by watching FDA inspectors take notes during an inspection. Have a new auditor observe a few audits before they are allowed to participate. Make sure they take notes and explain what you are doing and why they are observing as you conduct audits. Review the notes of new auditors periodically throughout the audit to provide suggestions for improvement and identify missing information. You can also record a supplier audit or internal audit and let a new trainee take notes on the pre-recorded webinar. This eliminates the need to coordinate schedules to involve the trainee.

Quality auditor training should include practicing audit agenda creation

Have new lead auditors submit a draft audit agenda to you before sending it to the supplier or department manager. Usually, the first audit agenda will need revision and possibly multiple revisions. Make sure you train the person to include enough detail in the agenda, and using a checklist or template is recommended. The agenda creation will be part of the audit preparation, and it can be done without time pressure.

How do you audit the auditing process?

Most quality managers are experienced and have little trouble planning an audit schedule. The next step is to conduct the audit. The problem is that there is very little objective oversight of the auditing process. The ISO 13485 standard for medical devices requires that “Auditors shall not audit their own work.” Therefore, most companies will opt for one of two solutions for auditing the internal audit process: 1) hire a consultant or 2) ask the Director of Regulatory Affairs to audit the internal auditing process.

Both of the above strategies for auditing the internal audit process meet the requirements of ISO 13485, but neither approach helps to improve an internal auditor’s performance. I have interviewed hundreds of audit program managers over the years, and the most common feedback audit program managers give is “Change the wording of this finding” or “You forgot to close this previous finding.” This type of feedback is related to the report-writing phase of the audit process. I rarely hear program managers explain how they help auditors improve at the other parts of the process.

When auditors are first being trained, we typically provide examples of best practices for audit preparation, checklists, interviewing techniques, AND reports. After auditors are “shadowed” by the audit program manager for an arbitrary three times, the auditors are now miraculously “trained.” Let’s see if I can draw an analogy to make my point.

That kind of sounds like watching your 16-year-old drive the family car three times and then giving them a license.

About the Author

Rob Packard 150x150 Seven ways to improve quality auditor trainingRobert Packard is a regulatory consultant with 25+ years of experience in the medical device, pharmaceutical, and biotechnology industries. He is a graduate of UConn in Chemical Engineering. Robert was a senior manager at several medical device companies—including the President/CEO of a laparoscopic imaging company. His Quality Management System expertise covers all aspects of developing, training, implementing, and maintaining ISO 13485 and ISO 14971 certifications. 2009-2012, he was a lead auditor and instructor for one of the largest Notified Bodies. Robert’s specialty is regulatory submissions for high-risk medical devices, such as implants and drug/device combination products for CE marking applications, Canadian medical device applications, and 510(k) submissions. The most favorite part of his job is training others. He can be reached via phone at 802.258.1881 or by email. You can also follow him on LinkedIn, Twitter, and YouTube.

Seven ways to improve quality auditor training Read More »

Audit schedule and an audit agenda, what’s the difference?

8 Comments / ISO Auditing / By / , ,

Internal audit and supplier audit programs both require an audit schedule and an audit agenda, but what’s the difference between them?

What is an audit schedule?

An “audit schedule” is not a formal definition in ISO 19011:2018. However, section 5.1 of that standard states that your audit program should include nine different requirements. Item “d” is “d) schedule (number/duration/frequency) of the audits.” Typically, the audit program manager will maintain an annual audit schedule with a date indicating the date the schedule was last revised. The most common example in lead auditor training is a matrix like the one shown below. The left-hand column will list all of the individual processes that are identified in the company’s process interaction diagram, and the top row of the matrix will indicate the month when each process audit will be conducted. Typically, the expectation is to complete the audit sometime during that month, but some quality auditing procedures specify that the audit may be completed the month before or the month after to accommodate the process owner. The regulations only require that you document and maintain an audit schedule, and the standard is only considered guidance.

%name Audit schedule and an audit agenda, what’s the difference?
Example of an Internal Audit Schedule

I use a slide in lead auditor courses that gives the example of an internal auditing schedule provided above. On the surface, this example seems like a good audit schedule. Twelve auditors perform two audits each year. If each audit requires approximately two days, each auditor spends less than two percent of their work year auditing. Unfortunately, a two-percent allocation of time is insufficient to become or remain proficient at auditing. An improvement to the auditing schedule would be to assign fewer auditors so each auditor gets more experience. There is no perfect number, but assigning a few specialists will improve the chances of becoming and remaining proficient at auditing.

What is an audit agenda?

An “audit agenda” is not a formal definition in ISO 19011:2018 either. In fact, the word “agenda” is not even used in ISO 19011. Instead, section 5.5.5 of ISO 19011 states that “The assignment [of the individual audit] should be made in sufficient time before the scheduled date of the audit, in order to ensure the effective planning of the audit.” The audit plan must also be part of the records [i.e., Clause 5.5.7(b)]. Therefore, “agenda” and “plan” may be used interchangeably. Details of audit planning are provided in 6.3.2 of ISO 19011.

6 Steps to Creating an Audit Schedule

There are six steps to creating an audit schedule:

  1. What were the results of previous audits?
  2. Which processes are the most important to audit?
  3. Who should conduct your internal audit?
  4. How long should your internal audit be?
  5. Should you conduct one full quality system audit or several audits?
  6. Is a remote audit good enough?

We will address each of the six steps below.

How do the results of previous audits impact your audit schedule?

The results of an audit include nonconformities, observations for improvement (OFI), and a conclusion regarding whether the quality system is effective or not. Usually, most processes are effective, and there are no nonconformities or OFIs. Therefore, any processes that had a nonconformity or OFI should be prioritized in the audit schedule and audit planning for the future. For internal and supplier audits, a best practice is for the auditor and the process owner to discuss the corrective actions planned and determine the appropriate timeline or implementation of actions planned. Then the auditor can indicate a timeframe for re-auditing the nonconforming process after corrective actions are implemented. This strategy allows the auditor to be part of the effectiveness check. This approach is appropriate for individual process audits but not for a full-quality system audit.

Which processes are the most important to audit?

The primary element impacting the importance of processes is the risk to product quality associated with the process. Usually, support processes are of lower importance because they do not directly impact product quality. In contrast, core processes directly involved in a device’s design, manufacture, and distribution are critical. Most auditors and audit program managers emphasize design controls and production process controls as important areas to audit. However, the distribution area is often neglected. Other core processes are purchasing, sales, customer service, and servicing. Not every process is equally important when comparing two companies. For example, device manufacturers that only make software as a medical device (SaMD) often have very limited purchasing and incoming inspection activities to audit.

Who should the audit program manager assign to each internal audit?

%name Audit schedule and an audit agenda, what’s the difference?The example of a revised audit schedule provided above identifies the departments where each of the auditors works with color coding. This is done to ensure that auditors are not assigned to audit processes where they might have a conflict of interest (i.e., they would be auditing their own work). This is the most important aspect of assigning auditors. The second most important aspect is to make sure the auditor has the technical knowledge to audit the process. It is challenging to conduct an audit of manufacturing if you have not spent any time in manufacturing before. If auditors are new and their training is in progress, then the audit program manager may assign the auditor to a process specifically to give them more experience with that type of process. Inexperienced auditors often are assigned less important processes that have not changed recently. However, a better approach to training auditors is to give them a challenge with support. Having the new auditor prepare a detailed sampling plan and list of questions before the audit can prepare them for auditing a more challenging, important process that is likely to have one or more nonconformities. Auditing processes that have nonconformities is also the best way to teach a new auditor how to write the audit findings.

What should be the duration of each internal audit in your schedule?

The duration of an audit should be based on the results of previous audits, but other important factors include: 1) the number of personnel involved in the process, 2) the complexity of the process, and 3) the risk to product quality associated with the process. The MDSAP program uses a procedure for audit time determination (i.e., MDSAP AU P0008.007: Audit Time Determination Procedure), and the MDSAP audit approach document (i.e., MDSAP AU P0002.008 Audit Approach) classifies processes as having either a “direct” or “indirect” impact upon product quality based upon the applicable clause of the process (i.e., Clauses 0-6.3 are indirect, and Clauses 6.4-8.5.3 are direct). For example, the production processes and design and development processes both involve a large number of people in most organizations, the processes are complex, and both processes directly impact product quality. Therefore, I typically allocate 3-4 hours to each of those processes during an audit. In comparison, incoming inspection often involves one or two people, and the process often involves only one procedure. Incoming inspection is a “direct” process, but less time (e.g., 1 hour) should be allocated to auditing incoming inspection than the other two processes–unless there was a nonconformity in the incoming inspection process during a recent audit or unless the process was recently changed.

Should you conduct one full quality system audit or several audits?

Both approaches have strengths and weaknesses, but there is not a single best way. If I am using employees to conduct an audit, then I typically restrict the scope of the audit to a single process. Alternatively, when I use a consultant to conduct an audit, I typically conduct a full-quality system audit to minimize travel costs. Another strategy I have recommended is to identify the processes that are most important to audit first (e.g., processes with recent changes and/or nonconformities), and these processes are scheduled for individual process audits during the first half of the audit schedule. Then I schedule a full-quality system audit in the second half of the audit schedule. The strategy ensures that all important processes will be audited twice in one year, but every process will be audited at least once.

Remote audits vs On-site audits

Prior to the Covid-19 pandemic, remote audits were rare in the medical device industry. Many NBs insisted that remote audits were not permitted or effective. The pandemic forced the entire industry to create policies for remote auditing and to use remote auditing whenever possible. Now that the pandemic has ended, many companies continue to conduct remote audits to save money. Even NBs are conducting more remote audits for Stage 1 readiness audits during the ISO 13485 certification process. ISO 19011 has a section in the Appendices outlining the differences between remote and on-site audits. However, there is a minimal advantage to conducting an on-site audit of a process where the auditor is expected to spend all of their time in a conference room during the audit. If the audit is going to be done in a conference room, then why not conduct it remotely? The one exception is when most records are paper-based and unavailable electronically. Alternatively, an on-site audit is generally more effective if the process involves observing inspection activities or assembly operations. Remote audits of inspection activities and assembly operations should be reserved for re-auditing or when a process has been audited on-site in the past, but an on-site audit would still be more effective for those processes.

How many times should a process be audited annually?

Many notified bodies will expect companies to audit all processes at least once during the year. However, it doesn’t expressly state this as a requirement in the regulations, and some companies justify skipping processes that are functioning well and have not changed in the past year. Our team is seeing this more frequently as the number of lead auditors worldwide has become scarce due to the requirements of MDSAP, the MDR/IVDR implementation, and unannounced audits. However, I almost never see the opposite justification (i.e., auditing a process more than once a year). If a process has been changed significantly, or there were nonconformities, then re-auditing the process may be used to verify the effectiveness of corrective actions or to verify that personnel are compliant with the revised process.

How to take advantage of the process approach to auditing

Another improvement that can be made to the revised example of an audit schedule is to use the process approach to auditing. Instead of performing an independent document control and training audit, these two clauses/procedures can be incorporated into every audit. The same is true of maintenance and calibration support processes. Wherever maintenance and calibration are relevant, these clauses should be investigated as part of auditing that area. For example, when the incoming inspection process is audited, it makes sense to look for evidence of calibration for any devices used to perform measurements in that area. When production process controls are being audited, maintenance records of production equipment should also be sampled.

If the concept of process auditing is fully implemented, the following ISO 13485 clauses can easily be audited in the regular course of reviewing other processes:

  • 4.2.1), Quality System Documentation;
  • 4.2.3), Document Control;
  • 4.2.4), Record Control;
  • 5.3), Quality Policy;
  • 5.4.1), Quality Objectives;
  • 6.2.2), Training;
  • 6.3), Maintenance;
  • 6.4), Work Environment;
  • 7.1), Planning of Product Realization & Risk Management
  • 7.6), Calibration;
  • 8.2.3), Monitoring & Measurement of Processes
  • 8.5.2), Corrective Action; and
  • 8.5.3) Preventive Action.

This strategy reduces the number of process audits needed by more than half.

Internal Auditing: Upstream/Downstream Examples

Another way to embrace the process approach to auditing is to assign auditors to upstream or downstream processes in the product realization process from their own area. For example, Manufacturing can audit Customer Service to understand better how customer requirements are confirmed during the order confirmation process. This is an example of auditing upstream because Manufacturing receives the orders from Customer Service—often indirectly through an MRP system. Using this approach allows someone from Manufacturing to identify opportunities for miscommunication between the two departments. If Regulatory Affairs audits the engineering process, this is an example of auditing downstream. Regulatory Affairs is often defining the requirements for the Technical Files and Design History Files that Engineering creates. If someone from Regulatory Affairs audits these processes, the auditor will realize what aspects of technical documentation are poorly understood by Engineering and quickly identify retraining opportunities.

Audit schedule and an audit agenda, what’s the difference? Read More »

Incoming Inspection – How to perform a single process audit

1 Comment / ISO Auditing / By

The incoming inspection process is my favorite process to audit, and it is the best process for teaching new auditors.

The above video demonstrates how to use a turtle diagram to conduct a process audit of the receiving inspection process. However, this article goes into more detail. You will learn what to look at and what to look for in each part of the audit process.

Preparation for your audit of incoming inspection

If you are conducting an audit of an incoming inspection, you will need a copy of the procedure (i.e., Receiving Inspection Procedure, SYS-033).

Receiving Inspection Procedure Image Incoming Inspection How to perform a single process audit

Do you need an opening meeting?

Opening meetings are not required for first-party (i.e., internal) and second-party (i.e., supplier) audits. Only third-party auditors are required to have a formal opening meeting. Having an opening meeting is always a good idea, but keep it brief and use a checklist. Try to set the tone for the audit with your opening meeting. This will be your second impression because you already had a conversation with the process owner in preparation for the meeting. However, you want to give everyone present for the opening meeting that you exhibit all the personality characteristics of a good auditor as defined by ISO 19011:2018. Professionalism, organization, and integrity should be obvious to everyone in the room. However, don’t forget to smile and be polite because your auditee might be very nervous. FDA inspectors seem to have an unwritten rule book (i.e., in addition to QSIT) that encourages them to intimidate the companies they inspect.

Step 1 – “Briefly, please describe the incoming inspection process.”

The purpose of this section is not to duplicate the level of detail found in the procedure. It is meant to provide a brief description of the process. Ideally, you want to write a single sentence for the incoming inspection process’s what, where, when, who, and how. A maximum of five sentences is needed to answer those five questions. The process owner should provide the description, and there is no need for them to go into extreme detail because you have at least six more questions to ask (see steps 2-7 below). If you are doing a supplier audit or an audit of a company you don’t work for, you might want to have a few “ice breaker” questions that precede this question. For example, you might ask the person’s name, title, and the number of years they have worked for the company. You might also consider stealing my favorite auditor disclaimer, “If you see me writing furiously, don’t worry. I’m required to write down objective evidence supporting conformity with requirements. If I start asking the same question three different ways, and I’m not writing any notes, that means I am having trouble finding evidence of conformity, and I need your help.”  

Step 2 – “What are the inputs that trigger incoming inspection?”

Inputs and outputs of any process refer to both information and physical items. For 100% administrative processes, you may not have any physical items. Incoming inspection, however, has physical goods you receive from suppliers and inspecting. Therefore, the process inputs you are looking for are physical goods and quality system records associated with those goods. For example, if a bunch of titanium round bars were ordered by a buyer in your purchasing department, the physical goods are the titanium bars. The purchase order is one of the quality system records. Other input records that are usually requested to be shipped with the titanium include a packing slip, a certification of analysis, and a dimensional inspection report. It is common to see the incoming inspection activity be delayed because the records are not included with the shipment from the supplier. One recommendation for a process improvement is to require the supplier to send records electronically at the time of shipment instead of sending hardcopies with the product. Statistical inspection sampling plans and work instructions are often confused with input records. These documents are needed to start the incoming inspection, but these are documents that belong in step six of the turtle diagram.

Step 3 – “What are the outputs of the incoming inspection process?”

After incoming inspection is completed there is a requirement to identify the status of the physical product (i.e., accepted or rejected). Usually, a green tag will be used to identify the product as accepted. The tag will also identify the part number, lot, and quantity of product accepted. If the product is titanium, each bar will get a tag. The product will then be transferred to a designated storage area. If you are conducting an audit of a supplier, or a full quality system audit, auditing the warehouse for storage and handling processes is a logical next process. The auditor should look for whether product is segregated in designated locations for specific types of product or if the storage locations are “random” but identified electronically in a material resource planning (MRP) system. The quality system records output from the incoming inspection process will be inspection records and either a green release tag or red rejection tag. If the product is rejected, the product shall be transferred to a quarantine area for nonconforming product and a nonconforming material record (i.e., NCMR) is initiated. Therefore, the process for controlling nonconforming material is another process that could be a logical next process to audit.

Step 4 – “What resources are needed for this process?”

This part of the process approach to auditing is one of the most neglected parts of the quality system. Resources include the facility infrastructure, manufacturing equipment, measurement devices used for inspection, and quality system software used to maintain records of incoming inspection. In this part of the process audit the auditor must be observant. Maintenance records might be located on the side of equipment and they can be reviewed as the auditor walks through the area. This would be an opportunity to interview personnel to make sure they can explain the maintenance process and the equipment maintenance is being performed as planned. The auditor should also determine if equipment validation is required. If the equipment is automated (e.g., automated optical inspection), then an installation qualification (i.e., IQ) should be requested as a quality system record to review at the end of the process or as part of the process for process validation. If the inspection area includes a metrology lab, then the environment may be temperature and humidity controlled. In these types of environments, records of environmental monitoring and trending of environmental conditions should be verified. Lighting, magnification, and particulate filtration could be other environmental requirements for the inspection area. Pest control should be verified in the receiving area, inspection area, and storage areas. The receiving area and warehouse storage are common areas to find pests. Calibration identification should be recorded as a potential follow-up trail for any measurement devices used in the inspection area, and if software is used you will want to verify that quality system software tool validation has been performed.

Step 5 – “Who performs this process?”

A combination of three different roles and responsibilities are typical for this process: 1) department manager, 2) receiving personnel, and 3) inspection personnel. Sometimes one or more of these roles will be combined into one job. The activities sometimes are only performed for a few hours each day, and the personnel that perform the incoming inspection process are assigned to other roles, such as warehouse storage, handling, and shipping. Auditors should always try to interview one or more of the people doing the receiving and inspection activities instead of limiting the interviews to the process owner. Often I will ask the personnel to demonstrate the receiving process and the inspection process. In order to make sure this is possible, you will need to communicate that you want to observe these activities prior to the audit or during the opening meeting. If you don’t, the receiving and inspection activities may already be completed before you start to interview the personnel. Any personnel that are unable to explain the tasks they perform may be targets for verification of training records, effectiveness of training, and competency.

Step 6 – “How is this process performed?”

If an auditor interviews personnel, most people will describe the process in a very haphazard way and steps will be missed. This is why asking people to demonstrate the process is better. The best method is for the person to access the current, approved work instruction or procedure for the process. Then the person should follow the work instruction step-by-step. This allows the person to use the work instruction or procedure as a “crutch” and reduces their nervousness. This also eliminates the skipping steps if the procedures and work instructions are sufficiently detailed. Any blank forms used and statistical inspection standards are also considered quality system documents that define how the process is performed. Sometimes the process owner will provide these documents during their interview, and other times this documents are provided as audit preparation documents. If the documents are not provided in advance the auditor should make sure that they review the documents during observation of activities being performed. This is where an auditor may identify the use of obsolete quality documents, missing details in the documents, and details that are inconsistently followed by personnel.

Step 7 – “What metrics are important for this process?”

Whenever I ask, “What metrics are important in this process?” I typically get a blank stare. Hundreds of business management leaders subscribe to the concept of “what gets managed gets done.” You are also required to establish metrics for your quality system processes in accordance with Clause 8.2.5. Therefore, you need to establish at least one metric, if not more than one. Auditing can help identify opportunities for improvement (OFI), but metrics are the best source of OFIs for a quality system. 

Do you need a closing meeting?

You should always conduct a closing meeting for your audits. However, it is also a best practice to summarize your findings for the process owner before you move on to the next process. If some records remain to be reviewed, ensure the process owner knows that the audit results are pending an outcome of reviewing the remaining records. Consider adopting the “sandwich” approach to presenting your findings: 1) something positive, 2) any nonconformities, and 3) something positive. The approach sandwiches the “bad news” between two pieces of “good news.” If you are working as part of a team, the lead auditor should always be aware of the results of your audit. The manager responsible for the process (i.e., the process owner) should also be aware of the results. Do everything you can to prevent unpleasant surprises at the end of the audit.

When you describe any nonconformities, make sure that you include all of the following information:

  1. the grading of the finding (i.e., MDSAP scoring or Major/Minor)
  2. a single sentence stating the finding
  3. the requirement, including a reference to the applicable regulation or standard
  4. objective evidence from your notes

Whenever possible, email a draft of the wording for your nonconformities to the process owner so they can be prepared with clarification questions during the closing meeting. Make sure you agree with your lead auditor before sending the wording of the finding, and copy them on the email communication. If the process owner has initiated immediate corrective action(s), make sure you note this in your report.

Finalizing your audit report

If you are conducting a supplier audit, you need to give the supplier formal feedback from the audit. You will need an audit report for your quality system records, but you are not required to give the supplier the full report. You might provide a summary of the audit for the supplier instead. If you do this, you should include a copy of that communication in your quality system record (e.g., an appendix to your audit report). If you are going to provide a summary of findings, the content should include at least the following:

  1. positive findings (i.e., strengths)
  2. negative findings (i.e., weaknesses)
  3. nonconformities (if any)
  4. required actions (e.g., supplier corrective action plan)
  5. due date(s) for objective evidence of containment, corrections, and corrective actions
  6. recommendations for follow-up (e.g., next audit)

If you prepare an internal audit report, all of the above content should be included. However, the report should have additional details:

  1. audit purpose
  2. audit scope
  3. audit date(s)
  4. audit criteria
  5. name of participants
  6. date of report
  7. closure of previous audit non-conformities
  8. reference to the audit agenda
  9. deviations, if any, from the agenda
  10. summary of the audit, including any obstructions
  11. objective evidence sampled (i.e., what you looked at and what you looked for)
  12. opportunities for improvement (if any)

Incoming Inspection – How to perform a single process audit Read More »

ISO 19011 – Do you need this quality system auditing standard?

5 Comments / ISO Auditing / By / , ,

Read this article to learn why ISO 19011 standard is a vital guidance for anyone that audits quality systems or manages an audit program.

What is ISO 19011?

ISO 19011 is a seven-part international standard for auditing management systems. The standard defines the eight principles of auditing (e.g., the process approach to auditing), provides guidance on managing audit programs and conducting audits, and includes recommendations for evaluating people for competency. There is also an appendix with details on conducting on-site and remote audits.

If you have ever taken a lead auditor course for ISO 13485, or one of the other quality management system standards, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Quality Management Systems.” In 2018, ISO 19011 was updated, and the changes were not superficial. If you need to purchase a copy of ISO 19011:2018, the Estonian Center for Standardization and Accreditation is the least expensive source we know.

ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting internal and external audits, and determining auditor competency.  One of the most common points of confusion in the lead auditor course is the difference between first, second, and third-party audits. In the first edition of this Standard, the difference between first, second, and third-party audits was just a note at the bottom of page one and the top of page two. The note was also not clear. In the second edition of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear. Table 1 was modified further in the 3rd edition to include a bottom row that remains unchanged in the 3rd edition, released in 2018.

Types of Audits Table 1 1024x205 ISO 19011 Do you need this quality system auditing standard?

Figure 1, found in Clause 5.1 of the 2nd edition, was combined with Figure 2, found in Clause 6.1 of the 2nd edition. The combined figure is now Figure 1 in the 3rd edition. The combined scope of Figure 1 is now a “Process flow for the management of an audit program” and a “Process flow for conducting an audit.” The figure categorizes the various stages of audit program management and conducting an audit into the Plan-Do-Check-Act (PDCA) cycle. We highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard.

The 2018 version still includes an opening meeting checklist (i.e., Clause 6.4.3) and a closing meeting checklist (i.e., Clause 6.4.10). Figure 3 in the 2nd edition, “Overview of the process of collecting and verifying information,” was a poor example of a flow chart. The committee did not update the figure when the standard was updated for the 3rd edition. Therefore, we updated the figure below to provide additional traceability to the Clauses of the Standard. If you incorporate this figure into your quality auditing procedure, you should substitute references to your procedure’s sections instead of the clauses of the standard.

Figure 2 ISO 19011 2018 1024x702 ISO 19011 Do you need this quality system auditing standard?

Competency Requirements in ISO 19011

Many audit procedures neglect to define the qualifications and methods for determining the competency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures we read include qualifications for a “Lead Auditor,” but we seldom see anything regarding competency. Unfortunately, this Standard only explicitly addresses the “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When we teach people how to be Lead Auditors, we spend more than an hour on this topic alone.

The Standard would be more effective by providing an example of how third-party auditors become qualified as a Lead Auditor. Third-party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meetings, conducting the audit, closing meetings, and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e., – Stage 2 certification or re-certification), and another qualified lead auditor must evaluate you and provide feedback.

Appendices in ISO 19011

The appendices were the last significant additions to this Standard in 2011 (i.e., 2nd edition). Annex A provided examples of discipline-specific knowledge and skills of auditors. This section was eliminated from the 3rd edition of ISO 19011:

“Due to the large number of individual management system standards, it would not be practical to include competence requirements for all disciplines.” – Copied from the Foreward

I think providing adding a short Annex to each management system standard that defines recommended discipline-specific knowledge would be helpful. Still, that kind of change would need to be initiated with the next version of ISO 9001.

Appendix B in the 2nd edition is now Appendix A in the 3rd edition of ISO 19011. A table (Table A.1 – Audit Methods) compares conducting on-site and remote audits. We were pleased to see that conducting interviews is a significant part of remote auditing in this table. Section A.17 in the appendix provides suggestions for conducting interviews. Still, if you exhibit all 13 professional behavior traits found in Clause 7.2.2, you don’t need advice on speaking with people. For the rest of us mortals, we could use a five-day course on interviewing alone. To improve your skills in this area, ask an experienced auditor with solid interviewing skills to watch and comment on a recording of a virtual audit you perform. Watching yourself audit is cringe-worthy, but we guarantee you will improve.

What are the primary changes to the 2018 version of the standard?

There are seven main differences between the second edition, published in 2011, and the third edition of ISO 19011, released in 2018:

  1. addition of a seventh principle of auditing in sub-clause 4(g) (i.e., risk-based approach);
  2. more guidance on audit program management in Clause 5, including audit program risk;
  3. expansion of Clause 6 on conducting an audit–especially Clause 6.3 on audit planning;
  4. expansion of auditor competence requirements in Clause 7;
  5. updating of terminology to emphasize processes rather than objects;
  6. removal of an annex containing competence requirements for specific quality management systems;
  7. expansion of Annex A to include guidance on new auditing concepts such as remote audits.

Risk-based auditing is the most significant change in the 2018 version of ISO 19011

One of the main differences between ISO 19011:2018 and the previous 2011 version is the addition of a “risk-based approach” to the principles of auditing. Specifically, clause 4(g) of the guidelines for auditing management systems is, “The risk-based approach should substantively influence the planning, conducting and reporting of audits to ensure that audits are focused on matters that are significant for the audit client, and for achieving the audit program objectives.” A lot of people are unsure of what is meant by a risk-based approach. Still, the key to understanding this is to focus on the definition of risk. From a product perspective, the risk is the “combination of the probability of occurrence of harm and the severity of that harm.” From a process perspective, the risk is the “effect of uncertainty on an expected result” (ISO 9001:2015, clause 3.09). Therefore, auditors should emphasize medical devices with the highest severity of harm and devices with a high probability of hazards or hazardous situations. When an auditor focuses on a process rather than a specific medical device, auditors should emphasize any processes that are not under control and any recent process changes.

animal nature reptile animal world ISO 19011 Do you need this quality system auditing standard?

What is risk-based auditing?

Risk-based auditing considers the risks of failing to achieve audit objectives and the opportunities created by choosing various audit methods and strategies. For example, a desktop audit of procedures might be appropriate if you are conducting your first internal audit for a new quality system. Alternatively, a desktop audit would be a waste of time if you are auditing a mature quality system where very few changes to procedures have been made in the past year. Using the element approach to auditing is unlikely to add much value. Audits are meant to be a sampling. Therefore, you should focus on areas of importance where previous nonconformities were identified, any new products or processes, and anything that changed significantly.

Auditor selection should also be risk-based

Suppose you are conducting a supplier audit as part of your initial supplier qualification for a critical component supplier or contract manufacturer. In that case, you should consider doing a team audit with a multi-disciplinary team. This is a risk-based approach to the supplier qualification process, which ensures that subject matter experts evaluate each process instead of auditors with a general quality assurance background. This approach also forces more of your personnel to introduce themselves to the new supplier, and the audit will develop more reliable communication channels between your two companies. Alternatively, if you are conducting a routine internal audit of a production process, you might select a new lead auditor to conduct the audit. You don’t expect any significant findings in a routine internal audit of an established production process. In your role as an audit program manager, you need to match the new lead auditor to a process that will force them to look at all aspects of the process approach to auditing. Specifically, process validation, calibration, maintenance, and process monitoring may not apply to other administrative process areas, such as purchasing.

Risk-based auditing should influence your auditing schedule

The frequency of auditing suppliers and internal process areas should reflect the associated risks. Therefore, when you create or update your auditing schedule, you should consider the risk level of the products being audited and the process being audited. Production processes with a moderate or high level of non-conforming products may need to be audited more than once yearly. Still, a supplier with an excellent track record of extremely high quality and on-time delivery may be audited in alternating years. If you previously scheduled a remote audit, you may want to alternate to conducting an on-site audit the next time.

The duration of your audits should not always be the same either. Suppose one production process makes one product in low volume, and another production process makes multiple products in high volume. In that case, you should not schedule a two-hour internal audit for both processes every year. The low-volume production process may only need a one-hour audit once per year. In contrast, the high-volume process may require a four-hour internal audit or multiple annual audits.

Risk-based auditing applied to remote supplier auditing

The risk-based auditing approach was added to ISO 19011:2018 as the seventh principle of auditing. This represents the most significant change to that standard, but how does it apply to remote auditing? Despite the opportunities created by remote auditing, there are also risks associated with auditing suppliers remotely. People worry about auditees hiding hazardous situations or unacceptable environmental conditions such as filth or disrepair. However, unacceptable cleanliness and maintenance practices don’t happen overnight. Therefore, you should expect a clean and well-maintained facility to remain that way. One approach is to alternate between remote and on-site audits to verify the overall condition of a supplier’s facility. Therefore, the risk of auditees hiding objective evidence is more an issue of trust than a highly probable occurrence.

The more probable risks associated with remote auditing are related to the potential lack of availability of records. This is especially important for paper-based quality systems. Most people try to address this risk by scanning paper documents and records, but scanning documents have limited value. Scanning paper documents is more efficiently performed in a large batch by an automated or semi-automated process. Also, auditors and inspectors typically focus on the most recent records, and auditors and inspectors rarely sample 100% of the records. Therefore, the best risk controls include the following:

  • Ask a guide to send a digital picture of the record.
  • Use a tripod-mounted HD webcam focused on a music stand or similar surface.
  • Ask the auditee to read the document while you take notes.

In our experience, you will probably rely on all three risk controls, but it is unlikely to delay the audit. However, in response to the limited physical access to medical device facilities and personnel, certification bodies are sending out questionnaires to assess the risk of being unable to achieve audit objectives or cover the required scope of surveillance and recertification audits. As the audit program manager, you can reduce these risks by working with supply chain managers to develop new supplier questionnaires that specifically ask questions about the capability of supporting audits remotely. In particular, it would be essential to obtain facility maps to identify areas with inadequate cellular coverage and identify records that are only available in hardcopy format.

ISO 19011 – Do you need this quality system auditing standard? Read More »

Scroll to Top