Auditing Archives

May 3 2023

Incoming Inspection – How to perform a single process audit

1 Comment / Auditing, Quality Management System / By Robert Packard

The incoming inspection process is my favorite process to audit, and it is the best process for teaching new auditors.

The above video demonstrates how to use a turtle diagram to conduct a process audit of the receiving inspection process. However, this article goes into more detail. You will learn what to look at and what to look for in each part of the audit process.

Preparation for your audit of incoming inspection

If you are conducting an audit of an incoming inspection, you will need a copy of the procedure (i.e., Receiving Inspection Procedure, SYS-033).

Do you need an opening meeting?

Opening meetings are not required for first-party (i.e., internal) and second-party (i.e., supplier) audits. Only third-party auditors are required to have a formal opening meeting. Having an opening meeting is always a good idea, but keep it brief and use a checklist. Try to set the tone for the audit with your opening meeting. This will be your second impression because you already had a conversation with the process owner in preparation for the meeting. However, you want to give everyone present for the opening meeting that you exhibit all the personality characteristics of a good auditor as defined by ISO 19011:2018. Professionalism, organization, and integrity should be obvious to everyone in the room. However, don’t forget to smile and be polite because your auditee might be very nervous. FDA inspectors seem to have an unwritten rule book (i.e., in addition to QSIT) that encourages them to intimidate the companies they inspect.

Step 1 – “Briefly, please describe the incoming inspection process.”

The purpose of this section is not to duplicate the level of detail found in the procedure. It is meant to provide a brief description of the process. Ideally, you want to write a single sentence for the incoming inspection process’s what, where, when, who, and how. A maximum of five sentences is needed to answer those five questions. The process owner should provide the description, and there is no need for them to go into extreme detail because you have at least six more questions to ask (see steps 2-7 below). If you are doing a supplier audit or an audit of a company you don’t work for, you might want to have a few “ice breaker” questions that precede this question. For example, you might ask the person’s name, title, and the number of years they have worked for the company. You might also consider stealing my favorite auditor disclaimer, “If you see me writing furiously, don’t worry. I’m required to write down objective evidence supporting conformity with requirements. If I start asking the same question three different ways, and I’m not writing any notes, that means I am having trouble finding evidence of conformity, and I need your help.”

Step 2 – “What are the inputs that trigger incoming inspection?”

Inputs and outputs of any process refer to both information and physical items. For 100% administrative processes, you may not have any physical items. Incoming inspection, however, has physical goods you receive from suppliers and inspecting. Therefore, the process inputs you are looking for are physical goods and quality system records associated with those goods. For example, if a bunch of titanium round bars were ordered by a buyer in your purchasing department, the physical goods are the titanium bars. The purchase order is one of the quality system records. Other input records that are usually requested to be shipped with the titanium include a packing slip, a certification of analysis, and a dimensional inspection report. It is common to see the incoming inspection activity be delayed because the records are not included with the shipment from the supplier. One recommendation for a process improvement is to require the supplier to send records electronically at the time of shipment instead of sending hardcopies with the product. Statistical inspection sampling plans and work instructions are often confused with input records. These documents are needed to start the incoming inspection, but these are documents that belong in step six of the turtle diagram.

Step 3 – “What are the outputs of the incoming inspection process?”

After incoming inspection is completed there is a requirement to identify the status of the physical product (i.e., accepted or rejected). Usually, a green tag will be used to identify the product as accepted. The tag will also identify the part number, lot, and quantity of product accepted. If the product is titanium, each bar will get a tag. The product will then be transferred to a designated storage area. If you are conducting an audit of a supplier, or a full quality system audit, auditing the warehouse for storage and handling processes is a logical next process. The auditor should look for whether product is segregated in designated locations for specific types of product or if the storage locations are “random” but identified electronically in a material resource planning (MRP) system. The quality system records output from the incoming inspection process will be inspection records and either a green release tag or red rejection tag. If the product is rejected, the product shall be transferred to a quarantine area for nonconforming product and a nonconforming material record (i.e., NCMR) is initiated. Therefore, the process for controlling nonconforming material is another process that could be a logical next process to audit.

Step 4 – “What resources are needed for this process?”

This part of the process approach to auditing is one of the most neglected parts of the quality system. Resources include the facility infrastructure, manufacturing equipment, measurement devices used for inspection, and quality system software used to maintain records of incoming inspection. In this part of the process audit the auditor must be observant. Maintenance records might be located on the side of equipment and they can be reviewed as the auditor walks through the area. This would be an opportunity to interview personnel to make sure they can explain the maintenance process and the equipment maintenance is being performed as planned. The auditor should also determine if equipment validation is required. If the equipment is automated (e.g., automated optical inspection), then an installation qualification (i.e., IQ) should be requested as a quality system record to review at the end of the process or as part of the process for process validation. If the inspection area includes a metrology lab, then the environment may be temperature and humidity controlled. In these types of environments, records of environmental monitoring and trending of environmental conditions should be verified. Lighting, magnification, and particulate filtration could be other environmental requirements for the inspection area. Pest control should be verified in the receiving area, inspection area, and storage areas. The receiving area and warehouse storage are common areas to find pests. Calibration identification should be recorded as a potential follow-up trail for any measurement devices used in the inspection area, and if software is used you will want to verify that quality system software tool validation has been performed.

Step 5 – “Who performs this process?”

A combination of three different roles and responsibilities are typical for this process: 1) department manager, 2) receiving personnel, and 3) inspection personnel. Sometimes one or more of these roles will be combined into one job. The activities sometimes are only performed for a few hours each day, and the personnel that perform the incoming inspection process are assigned to other roles, such as warehouse storage, handling, and shipping. Auditors should always try to interview one or more of the people doing the receiving and inspection activities instead of limiting the interviews to the process owner. Often I will ask the personnel to demonstrate the receiving process and the inspection process. In order to make sure this is possible, you will need to communicate that you want to observe these activities prior to the audit or during the opening meeting. If you don’t, the receiving and inspection activities may already be completed before you start to interview the personnel. Any personnel that are unable to explain the tasks they perform may be targets for verification of training records, effectiveness of training, and competency.

Step 6 – “How is this process performed?”

If an auditor interviews personnel, most people will describe the process in a very haphazard way and steps will be missed. This is why asking people to demonstrate the process is better. The best method is for the person to access the current, approved work instruction or procedure for the process. Then the person should follow the work instruction step-by-step. This allows the person to use the work instruction or procedure as a “crutch” and reduces their nervousness. This also eliminates the skipping steps if the procedures and work instructions are sufficiently detailed. Any blank forms used and statistical inspection standards are also considered quality system documents that define how the process is performed. Sometimes the process owner will provide these documents during their interview, and other times this documents are provided as audit preparation documents. If the documents are not provided in advance the auditor should make sure that they review the documents during observation of activities being performed. This is where an auditor may identify the use of obsolete quality documents, missing details in the documents, and details that are inconsistently followed by personnel.

Step 7 – “What metrics are important for this process?”

Whenever I ask, “What metrics are important in this process?” I typically get a blank stare. Hundreds of business management leaders subscribe to the concept of “what gets managed gets done.” You are also required to establish metrics for your quality system processes in accordance with Clause 8.2.5. Therefore, you need to establish at least one metric, if not more than one. Auditing can help identify opportunities for improvement (OFI), but metrics are the best source of OFIs for a quality system.

Do you need a closing meeting?

You should always conduct a closing meeting for your audits. However, it is also a best practice to summarize your findings for the process owner before you move on to the next process. If some records remain to be reviewed, ensure the process owner knows that the audit results are pending an outcome of reviewing the remaining records. Consider adopting the “sandwich” approach to presenting your findings: 1) something positive, 2) any nonconformities, and 3) something positive. The approach sandwiches the “bad news” between two pieces of “good news.” If you are working as part of a team, the lead auditor should always be aware of the results of your audit. The manager responsible for the process (i.e., the process owner) should also be aware of the results. Do everything you can to prevent unpleasant surprises at the end of the audit.

When you describe any nonconformities, make sure that you include all of the following information:

the grading of the finding (i.e., MDSAP scoring or Major/Minor)
a single sentence stating the finding
the requirement, including a reference to the applicable regulation or standard
objective evidence from your notes

Whenever possible, email a draft of the wording for your nonconformities to the process owner so they can be prepared with clarification questions during the closing meeting. Make sure you agree with your lead auditor before sending the wording of the finding, and copy them on the email communication. If the process owner has initiated immediate corrective action(s), make sure you note this in your report.

Finalizing your audit report

If you are conducting a supplier audit, you need to give the supplier formal feedback from the audit. You will need an audit report for your quality system records, but you are not required to give the supplier the full report. You might provide a summary of the audit for the supplier instead. If you do this, you should include a copy of that communication in your quality system record (e.g., an appendix to your audit report). If you are going to provide a summary of findings, the content should include at least the following:

positive findings (i.e., strengths)
negative findings (i.e., weaknesses)
nonconformities (if any)
required actions (e.g., supplier corrective action plan)
due date(s) for objective evidence of containment, corrections, and corrective actions
recommendations for follow-up (e.g., next audit)

If you prepare an internal audit report, all of the above content should be included. However, the report should have additional details:

audit purpose
audit scope
audit date(s)
audit criteria
name of participants
date of report
closure of previous audit non-conformities
reference to the audit agenda
deviations, if any, from the agenda
summary of the audit, including any obstructions
objective evidence sampled (i.e., what you looked at and what you looked for)
opportunities for improvement (if any)

Incoming Inspection – How to perform a single process audit Read More »

Jan 31 2023

ISO 19011 – Do you need this quality system auditing standard?

Read this article to learn why ISO 19011 standard is a vital guidance for anyone that audits quality systems or manages an audit program.

What is ISO 19011?

ISO 19011 is a seven-part international standard for auditing management systems. The standard defines the eight principles of auditing (e.g., the process approach to auditing), provides guidance on managing audit programs and conducting audits, and includes recommendations for evaluating people for competency. There is also an appendix with details on conducting on-site and remote audits.

If you have ever taken a lead auditor course for ISO 13485, or one of the other quality management system standards, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Quality Management Systems.” In 2018, ISO 19011 was updated, and the changes were not superficial. If you need to purchase a copy of ISO 19011:2018, the Estonian Center for Standardization and Accreditation is the least expensive source we know.

ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting internal and external audits, and determining auditor competency. One of the most common points of confusion in the lead auditor course is the difference between first, second, and third-party audits. In the first edition of this Standard, the difference between first, second, and third-party audits was just a note at the bottom of page one and the top of page two. The note was also not clear. In the second edition of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear. Table 1 was modified further in the 3rd edition to include a bottom row that remains unchanged in the 3rd edition, released in 2018.

Figure 1, found in Clause 5.1 of the 2nd edition, was combined with Figure 2, found in Clause 6.1 of the 2nd edition. The combined figure is now Figure 1 in the 3rd edition. The combined scope of Figure 1 is now a “Process flow for the management of an audit program” and a “Process flow for conducting an audit.” The figure categorizes the various stages of audit program management and conducting an audit into the Plan-Do-Check-Act (PDCA) cycle. We highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard.

The 2018 version still includes an opening meeting checklist (i.e., Clause 6.4.3) and a closing meeting checklist (i.e., Clause 6.4.10). Figure 3 in the 2nd edition, “Overview of the process of collecting and verifying information,” was a poor example of a flow chart. The committee did not update the figure when the standard was updated for the 3rd edition. Therefore, we updated the figure below to provide additional traceability to the Clauses of the Standard. If you incorporate this figure into your quality auditing procedure, you should substitute references to your procedure’s sections instead of the clauses of the standard.

Competency Requirements in ISO 19011

Many audit procedures neglect to define the qualifications and methods for determining the competency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures we read include qualifications for a “Lead Auditor,” but we seldom see anything regarding competency. Unfortunately, this Standard only explicitly addresses the “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When we teach people how to be Lead Auditors, we spend more than an hour on this topic alone.

The Standard would be more effective by providing an example of how third-party auditors become qualified as a Lead Auditor. Third-party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meetings, conducting the audit, closing meetings, and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e., – Stage 2 certification or re-certification), and another qualified lead auditor must evaluate you and provide feedback.

Appendices in ISO 19011

The appendices were the last significant additions to this Standard in 2011 (i.e., 2nd edition). Annex A provided examples of discipline-specific knowledge and skills of auditors. This section was eliminated from the 3rd edition of ISO 19011:

“Due to the large number of individual management system standards, it would not be practical to include competence requirements for all disciplines.” – Copied from the Foreward

I think providing adding a short Annex to each management system standard that defines recommended discipline-specific knowledge would be helpful. Still, that kind of change would need to be initiated with the next version of ISO 9001.

Appendix B in the 2nd edition is now Appendix A in the 3rd edition of ISO 19011. A table (Table A.1 – Audit Methods) compares conducting on-site and remote audits. We were pleased to see that conducting interviews is a significant part of remote auditing in this table. Section A.17 in the appendix provides suggestions for conducting interviews. Still, if you exhibit all 13 professional behavior traits found in Clause 7.2.2, you don’t need advice on speaking with people. For the rest of us mortals, we could use a five-day course on interviewing alone. To improve your skills in this area, ask an experienced auditor with solid interviewing skills to watch and comment on a recording of a virtual audit you perform. Watching yourself audit is cringe-worthy, but we guarantee you will improve.

What are the primary changes to the 2018 version of the standard?

There are seven main differences between the second edition, published in 2011, and the third edition of ISO 19011, released in 2018:

addition of a seventh principle of auditing in sub-clause 4(g) (i.e., risk-based approach);
more guidance on audit program management in Clause 5, including audit program risk;
expansion of Clause 6 on conducting an audit–especially Clause 6.3 on audit planning;
expansion of auditor competence requirements in Clause 7;
updating of terminology to emphasize processes rather than objects;
removal of an annex containing competence requirements for specific quality management systems;
expansion of Annex A to include guidance on new auditing concepts such as remote audits.

Risk-based auditing is the most significant change in the 2018 version of ISO 19011

One of the main differences between ISO 19011:2018 and the previous 2011 version is the addition of a “risk-based approach” to the principles of auditing. Specifically, clause 4(g) of the guidelines for auditing management systems is, “The risk-based approach should substantively influence the planning, conducting and reporting of audits to ensure that audits are focused on matters that are significant for the audit client, and for achieving the audit program objectives.” A lot of people are unsure of what is meant by a risk-based approach. Still, the key to understanding this is to focus on the definition of risk. From a product perspective, the risk is the “combination of the probability of occurrence of harm and the severity of that harm.” From a process perspective, the risk is the “effect of uncertainty on an expected result” (ISO 9001:2015, clause 3.09). Therefore, auditors should emphasize medical devices with the highest severity of harm and devices with a high probability of hazards or hazardous situations. When an auditor focuses on a process rather than a specific medical device, auditors should emphasize any processes that are not under control and any recent process changes.

What is risk-based auditing?

Risk-based auditing considers the risks of failing to achieve audit objectives and the opportunities created by choosing various audit methods and strategies. For example, a desktop audit of procedures might be appropriate if you are conducting your first internal audit for a new quality system. Alternatively, a desktop audit would be a waste of time if you are auditing a mature quality system where very few changes to procedures have been made in the past year. Using the element approach to auditing is unlikely to add much value. Audits are meant to be a sampling. Therefore, you should focus on areas of importance where previous nonconformities were identified, any new products or processes, and anything that changed significantly.

Auditor selection should also be risk-based

Suppose you are conducting a supplier audit as part of your initial supplier qualification for a critical component supplier or contract manufacturer. In that case, you should consider doing a team audit with a multi-disciplinary team. This is a risk-based approach to the supplier qualification process, which ensures that subject matter experts evaluate each process instead of auditors with a general quality assurance background. This approach also forces more of your personnel to introduce themselves to the new supplier, and the audit will develop more reliable communication channels between your two companies. Alternatively, if you are conducting a routine internal audit of a production process, you might select a new lead auditor to conduct the audit. You don’t expect any significant findings in a routine internal audit of an established production process. In your role as an audit program manager, you need to match the new lead auditor to a process that will force them to look at all aspects of the process approach to auditing. Specifically, process validation, calibration, maintenance, and process monitoring may not apply to other administrative process areas, such as purchasing.

Risk-based auditing should influence your auditing schedule

The frequency of auditing suppliers and internal process areas should reflect the associated risks. Therefore, when you create or update your auditing schedule, you should consider the risk level of the products being audited and the process being audited. Production processes with a moderate or high level of non-conforming products may need to be audited more than once yearly. Still, a supplier with an excellent track record of extremely high quality and on-time delivery may be audited in alternating years. If you previously scheduled a remote audit, you may want to alternate to conducting an on-site audit the next time.

The duration of your audits should not always be the same either. Suppose one production process makes one product in low volume, and another production process makes multiple products in high volume. In that case, you should not schedule a two-hour internal audit for both processes every year. The low-volume production process may only need a one-hour audit once per year. In contrast, the high-volume process may require a four-hour internal audit or multiple annual audits.

Risk-based auditing applied to remote supplier auditing

The risk-based auditing approach was added to ISO 19011:2018 as the seventh principle of auditing. This represents the most significant change to that standard, but how does it apply to remote auditing? Despite the opportunities created by remote auditing, there are also risks associated with auditing suppliers remotely. People worry about auditees hiding hazardous situations or unacceptable environmental conditions such as filth or disrepair. However, unacceptable cleanliness and maintenance practices don’t happen overnight. Therefore, you should expect a clean and well-maintained facility to remain that way. One approach is to alternate between remote and on-site audits to verify the overall condition of a supplier’s facility. Therefore, the risk of auditees hiding objective evidence is more an issue of trust than a highly probable occurrence.

The more probable risks associated with remote auditing are related to the potential lack of availability of records. This is especially important for paper-based quality systems. Most people try to address this risk by scanning paper documents and records, but scanning documents have limited value. Scanning paper documents is more efficiently performed in a large batch by an automated or semi-automated process. Also, auditors and inspectors typically focus on the most recent records, and auditors and inspectors rarely sample 100% of the records. Therefore, the best risk controls include the following:

Ask a guide to send a digital picture of the record.
Use a tripod-mounted HD webcam focused on a music stand or similar surface.
Ask the auditee to read the document while you take notes.

In our experience, you will probably rely on all three risk controls, but it is unlikely to delay the audit. However, in response to the limited physical access to medical device facilities and personnel, certification bodies are sending out questionnaires to assess the risk of being unable to achieve audit objectives or cover the required scope of surveillance and recertification audits. As the audit program manager, you can reduce these risks by working with supply chain managers to develop new supplier questionnaires that specifically ask questions about the capability of supporting audits remotely. In particular, it would be essential to obtain facility maps to identify areas with inadequate cellular coverage and identify records that are only available in hardcopy format.

ISO 19011 – Do you need this quality system auditing standard? Read More »

Nov 1 2022

Auditing Risk Management Files

3 Comments / Auditing, CE Marking, ISO 14971:2019 (Risk Management), ISO Auditing, Remote Auditing / By Robert Packard / Audit, ISO 14971

What do you look at and look for when you are auditing risk management files to ISO 14971 and the new Regulation (EU) 2017/745?

Your cart is empty

Next week, November 15th @ Noon EST, you will have the opportunity to watch a live webinar teaching you what to look at and what to look for when you are auditing risk management files to Regulation (EU) 2017/745 and ISO 14971. Risk Management Files are one of the essential requirements of technical documentation required for CE Marking of medical devices. Most quality system auditors are trained on how to audit to ISO 13485:2016 (or an earlier version of that standard), but very few quality system auditors have the training necessary to audit risk management files.

Why you are not qualified to audit risk management files

Being a qualified lead auditor is not enough to audit the risk management process. When you are auditing a risk management file, you need risk management training and lead auditor training. To audit the risk management process, you will also need training on applicable guidance documents (i.e., ISO/TR 24971:2020) and applicable regulations (i.e., Regulation 2017/745 and/or Regulation 2017/746). There may also be device-specific guidance documents that specify known risks and risk controls that are considered state-of-the-art.

Creating an audit agenda

Once you have scheduled an audit of risk management files, and assigned a lead auditor, then the lead auditor needs to create an audit agenda. The audit can be a desktop audit that is performed remotely, or it can be an on-site audit. Regardless of the approach, the audit should include interviewing participants in the risk management process documented in the risk management file. As a rule of thumb, I expect a minimum of 30 minutes to be spent interviewing the process owner and one or more other participants. Then I spend an additional 60 minutes of auditing time reviewing documents and records.

Your audit agenda should specify the following items at a minimum:

the method of auditing to be used,
date(s) of the audit,
the duration of the audit,
the location of the audit, and
the auditing criteria.

The auditor(s) and the auditee participants should be identified in the audit agenda. Finally, you should specify which documents and records are required for audit preparation. These documents will be used to help identify audit checklist questions and to determine a sampling plan for the audit. At a minimum, you will need a copy of the risk management procedure and a list of the risk management files that are available to audit. You may also want to request the audit plan for each of those risk management files.

What did you look at and look for during your risk management audit?

When you audit the risk management process, you could take any of the following approaches or a combination of more than one. You could audit the process according to the risk management procedure. You could audit the process according to the risk management plan(s) for each risk management file. You could audit using the process approach to auditing. Finally, you could audit in accordance with specific requirements in the ISO 14971:2019 standard and applicable regulations (i.e., Regulation 2017/745). Regardless of which approach you take, your audit notes and the audit report should identify which documents and records you sampled and what you looked for in each document. Providing only a list of the documents is not enough detail.

Creating an auditing checklist for risk management files

Auditors with limited experience are taught to create an audit checklist by creating a table that includes each of the requirements of the audit criteria. For a risk management file, this would include a list of each of the requirements in ISO 14971 for a risk management file (i.e., Clause 9???). However, this approach is more like the approach that you should be using for a gap analysis. The better approach for creating an audit checklist for risk management files is to start by creating a turtle diagram. In the “process inputs” section (i.e., step 2 of 7), you would add questions derived from your review of the risk management plan(s). In the “process outputs” section (i.e., step 3 of 7), you would add questions specific to the risk management report and other records required in a risk management file. In the “with whom” section (i.e., step 5 of 7), you would add questions related to training and competency. You might also identify additional people involved in the risk management process, other than the process owner, to interview as a follow-up trail. In the “how done” section (i.e., step 6 of 7), you would add questions specific to the procedure and forms used for the risk management process. Finally, in the “metrics” section (i.e., 7 of 7), you would verify that the company is conducting risk management reviews and updating risk management documentation in accordance with the risk management procedure and individual risk management plan(s).

Audits are just samples

Just because you can generate a lot of questions for an audit checklist does not mean that you are required to address every question. Audits are intended to be a “spot check” to verify the effectiveness of a process. You should allocate your auditing resources based on the importance of a process and the results of previous audits. I recommend approximately three days for a full quality system audit, and approximately 90-minutes should be devoted to a process unless it is the design control process (i.e., Clause 7.3 of ISO 13485) which typically requires three to four hours due to the importance and complexity of the design controls process. Therefore, you should schedule approximately 30 minutes to interview people for the risk management process and approximately 60 minutes should be reserved for reviewing documents and records. With this limited amount of time, you will not be able to review every record or interview everyone that was involved in the risk management process. This is why auditors always remind auditees that an audit is just a sampling.

Which records are required in a risk management file?

The contents of a risk management file is specified in ISO 14971:2019, Clause 4.5. There are only four bullets in that section, but the preceding sentence says, “In addition to the requirements of other clauses of this document.” Therefore, your risk management file should address all of the requirements in ISO 14971:2019. What I recommend is a virtual risk management folder for each risk management file. As the auditor, you should also request a copy of the risk management policy and procedure. An example of what this would look like is provided below. The numbers in front of each subfolder correspond to the sub-clause or clause for that requirement in ISO 14971:2019.

Which records are most valuable when auditing risk management files?

As an auditor, I typically focus on three types of targets when auditing any process. First, I will sample any corrective actions implemented in response to previous audit findings. Second, will sample documents and records associated with any changes made to the process. Changes would also include any changes that were made to individual risk management files or the creation of a new risk management file. Finally, my third target for audit sampling is any item that I feel is at risk for safety or performance failures. The severity of the safety or performance failure is also considered when prioritizing audit sampling. In the context of a risk management file, I always verify that production and post-production activities are being conducted as planned. I try to verify that risk analysis documentation was reviewed for the need to update the documentation in response to complaints and adverse events.

More auditor training on risk management files

We are recording a live webinar intended to teach internal auditors and consultants how to perform a thorough audit of risk management files against the requirements of the new European Regulation (EU) 2017/745 and ISO 14971.

Auditing Risk Management Files

In this new webinar, you will learn how to conduct a process audit of risk management files. You will learn what to look at and what to look for in order to verify compliance with Regulation (EU) 2017/745 and ISO 14971:2019. The webinar will be approximately one hour in duration. Attendees will be invited to participate in the live webinar and receive a copy of the native slide deck. Anyone purchasing after the live event will receive a link to download the recording of the live event and the native slide deck.

Price: $64.50

In addition to this webinar on auditing risk management files, we also have other risk management training webinars available. The webinar on auditing risk management files will be hosted live on November 15, 2022 @ Noon EST (incorrect in the live video announcement).

Auditing Risk Management Files Read More »

Jul 1 2020

Why remote audit duration should never exceed 90 minutes

1 Comment / Auditing, ISO Auditing, Remote Auditing / By Robert Packard

This article explains why remote audit duration should not exceed 90 minutes and the unique opportunities created by a series of short remote audits.

Parkinson’s Law and the subject of audit duration

On November 19, 1995, Cyril Northcote Parkinson published an essay in the Economist. The title of the article was “Parkinson’s Law.” In the first sentence of the essay, Parkinson says, “It is a commonplace observation that work expands to fill the time available for its completion.” This essay refers to the observation that work is elastic concerning the demands on time when completing paperwork. When I first trained as an auditor, trainers emphasized that the most significant challenge faced by auditors is to complete an audit within the time available. An auditor’s task is to achieve the audit objectives within the time specified by the audit program manager. Time is precious, and you cannot easily extend the audit duration after scheduling the audit.

How much time is needed for a full quality system audit?

This question is a silly question to ask a consultant that works on an hourly basis. A consultant working on an hourly basis will make more money if they work more hours. Therefore, there is little incentive to underestimate the time required to complete the objectives of an audit. However, after completing hundreds of audits, I can honestly state that eight hours is not enough time to perform a full quality system audit of a medical device company’s quality system. However, I completed a full quality system audit of a small company in less than two days. I also had difficulty completing an audit of a larger company in four days. An FDA inspector typically requires four days to complete a routine inspection, even at foreign manufacturers where English is a second language, and they only need to return on the fifth day to prepare their FDA 483 observations to give to the company. Therefore, three days is typically the absolute minimum time required to complete a full quality system audit.

Does Parkinson’s Law apply to audit duration?

Parkinson’s Law certainly applies to the audit duration. If the lead auditor assigns a team member to review the CAPA process, the task is unlikely to be completed in 30 minutes, and most auditors would struggle to appear busy for more than three hours. You need enough notes to provide objective evidence of conformity for your audit report, but if you finish too quickly, then others may perceive that you were not thorough. Therefore, most auditors will begin any process audit by asking for a copy of the procedure and a log of the records available. The auditor will quickly review the procedure’s revision history to determine when the last revision was made and if there have been any significant revisions since the last audit. Next, the auditor will review the log to estimate how many records should be sampled. The auditor will then estimate how much time is needed to review the sampled records. Finally, a quick mental calculation is made to determine how much time remains for procedure review before the auditor must move on to interview the next subject matter expert.

Why are auditors always behind schedule?

An auditor begins with small, close-ended questions that are designed to put the auditee at ease. The auditor may even comment on unrelated subjects to build rapport first. Records may not be readily available, but auditors almost always have to wait for record retrieval. The request is recorded, copies are made, and the subject matter expert may need a little time to review before handing the auditor the requested record. Auditors will ask clarifying questions, and auditees will need a few moments to check their facts. Any one of these delays is insignificant by itself, but collectively there may be two-and-half minutes of delay cumulatively for each record requested if you sample five records, which represents a combined delay 12.5 minutes. If you average only seven minutes to review each record, then a sampling of five records will require 47.5 minutes. This will leave you only 12.5 minutes for introductions, review of the procedure, and conclusions. If you want to interview any of the people that investigated root-cause, then you will need more than an hour to complete your audit, and you will not finish in the one hour scheduled.

Why is it so hard to complete a full quality system audit in three days?

Most of your process audits require a few more minutes than you expected, but you will also need time to walk to the next subject matter expert, or you will be waiting for the next subject matter expert to enter the conference room. If the quality system consists of only the minimum twenty-eight required procedures, your full quality system audit will require more than 28 hours to complete. If there are additional regulatory requirements for CE Marking or ISO 13485 certification, you will need even more time to audit every process. You should also expect certain processes to require more time to properly sample records, such as technical documentation and design controls. Even the most experienced auditors struggle to review a technical file and/or design history file in less than two hours.

What happens to an auditor after auditing all day?

As a Notified Body auditor, I used to leave my home in Vermont on Sunday afternoon and drive two hours to the nearest major airport. Then I would be gone all week conducting audits. On Friday, I would drive home and arrive in the middle of the night. Each day audits would begin early in the morning, and I would complete the day after 8.5 to 9 hours of work. Jet lag, sleep deprivation, too little exercise, and constantly eating at restaurants took its toll. I would consult my Google calendar to learn what city I was in each morning, and to remember what company I was on my schedule for the day. I would purposely try to do as much walking around during the day just to keep my blood flowing and to help stay awake. I would read documents while pacing back-and-forth in conference rooms, and I would always make sure that we had to audit the most remote area of a facility after lunch to make sure that I didn’t fall asleep. I will tell stories and jokes to entertain my hosts, but it was necessary to break up the monotony of auditing quality systems seven days a week. I would make sure I drank at least six liters of water each day for health, but this also gave me an excuse to go to take frequent bathroom breaks. Somehow I managed to survive that lifestyle for more than three years. Each day my feet, legs, back, and neck were in severe pain. I had constant headaches, and I know the quality of my work gradually declined throughout each day. The most valuable lesson I learned was, you need to move frequently, or you will die.

What happens when you sit in front of a computer for eight hours?

I can sit in front of a computer longer than almost anyone I know. When I focus on work, four hours can elapse without me getting up from a chair even once. I might pick up my empty coffee mug four or five times to take a sip before I am conscious of the need to get another cup. On days where my schedule consists primarily of Zoom meetings, I may sit through as many as six consecutive meetings before I take the time to get up and go to the bathroom and get a drink of water. Clients may perceive that I have tremendous endurance, but there are negative consequences to this work pattern. My wrist becomes sore, and I need to switch my mouse pad and the style of the mouse I am using every day. I change computers, switch microphones, and take a short walk. My neck, back, and legs will hurt worse than any of the audits during my years as a Notified Body auditor. Sitting at a computer all day has resulted in mild symptoms of restless legs syndrome. Sitting at a computer continuously for the audit duration is physically exhausting and tedious. If you must complete a remote audit on a continuous eight-hour day, you can, but it is not healthy or productive. The negative health consequences and negative impact on productivity are equally applicable to auditees.

What can you do to reduce audit fatigue during a remote audit?

The most straightforward strategy for reducing fatigue is to take breaks. Instead of auditing for eight hours continuously, try auditing in two or three 90-minutes segments each day. If you are auditing someone in a different time zone, you may only be able to accommodate an audit duration of one 90-minute session per day without working through the night. Taking breaks will allow you to leave your computer, eat food, and even go to the bathroom. You can recharge your headset during a break too. You should consider taking a walk outside. It is incredible how much better you feel when you get some exercise, stretch, and experience a little natural light instead of the unnatural glow of your computer’s monitor. The person you are auditing will appreciate the breaks, but they will also enjoy the improvement in your overall demeanor. A simple smile after a 30-minute break has a tremendous positive impact.

How can we utilize breaks more effectively during remote audits?

Auditors need documents and records to review as objective evidence. The most obvious way to make use of breaks is for the auditor to give the auditee a list of documents and records to gather during the break. This will give the auditee an excuse to go and get the documents and records if they are stored in another location. The auditee might also scan records during a break. A break also gives subject matter experts time to re-familiarize themselves with the documents and records before resuming the audit. Auditees and auditors will need to recharge batteries, but the auditor might take time to convert their notes into a summary for the final audit report. The auditor might also review the audit criteria one more time before writing a nonconformity. The auditee might take advantage of the break to initiate a new CAPA and write a draft of the corrective action plan. Then when the audit resumes, the auditee can review the draft plan with the auditor to ensure that the plan is appropriate and nothing was accidentally omitted from the CAPA plan.

Why are 90 minutes a magical audit duration?

Auditing one process in a single 45-60 minute session is ok, but if you audit two processes in a single 90-minute session, you can reduce the time spend starting and stopping the audit session by half. Adding a third process to a single session will have a smaller impact, and the meeting will need to be so long that most participants will begin to lose concentration, and fatigue becomes a significant factor. Ninety minutes is not quite long enough to audit two processes effectively. Still, an auditor can request procedures in advance of the session or spend time after the session reviewing procedures. Therefore, by paying an additional 30 minutes reviewing two procedures “off-line,” the auditor can dedicate 100% of the “on-line” time to reviewing records and interviewing subject matter experts. The result is a fast-paced, 90-minute session where each subject matter expert typically is only needed for 45 minutes. Alternatively, if you are auditing more complex records like a design history file, you can spend all 90 minutes discussing that area.

Why remote audit duration should never exceed 90 minutes Read More »

Aug 1 2019

Auditing Technical Files

5 Comments / Auditing, Technical Files / By Robert Packard

This article explains what to look at and what to look for when you are auditing technical files to the new Regulation (EU) 2017/745 for medical devices.

Your cart is empty

On August 8th, 2019, we recorded a live webinar teaching you what to look at and what to look for when you are auditing technical files (a link for purchasing the webinar is at the end of this article). Technical files are the technical documentation required for CE Marking of medical devices. Most quality system auditors are trained on how to audit to ISO 13485:2016 (or an earlier version of that standard), but very few quality system auditors have the training necessary to audit technical files.

Why you’re not qualified to auditing technical files

If you are a lead auditor, you are probably a quality manager or a quality engineer. You have experience performing verification testing and validation testing, but you have not prepared a complete technical file yourself. You certainly can’t describe yourself as a regulatory expert. You are a quality system expert. A couple of webinars on the new European regulations are not enough to feel confident about exactly what the content and format of a technical file for CE marking should be.

Creating an auditing checklist

Most auditors attempt to prepare for auditing the new EU medical device regulations by creating a checklist. The auditor copies each section of the regulation into the left column of a table. Then the auditor plans to fill in the right-hand columns of the table (i.e., the audit checklist), with the records they looked at and what they looked for in the records. Unfortunately, if you have never created an Essential Requirements Checklist (ERC) before, you can only write in your audit notes that the checklist was completed and what the revision date is. How would you know if the ERC was correctly completed?

In addition to the ERC, now called the Essential Performance and Safety Requirements (i.e., Annex I of new EU regulations), you also need to audit all the Technical Documentation requirements (i.e., Annex II), all the Technical Documentation on Post-Market Surveillance (i.e., Annex III), and the Declaration of Conformity (i.e., Annex IV). These four annexes are 19 pages long. If you try to copy and paste each section into an audit checklist, you will have a 25-page checklist with more than 400 things to check. The result will be a bunch of checkboxes marked “Yes,” and your audit will add no value.

Audits are just samples

Every auditor is trained that audits are just samples. You can’t review 100% of the records during an audit. You can only sample the records as a “spot check.” The average technical file is more than 1,000 pages long, and most medical device manufacturers have multiple technical files. A small company might have four technical files. A medium-sized company might have 20 technical files, and a large device company might have over 100 files. (…and you thought the 177-page regulation was long.)

Instead of checking many boxes, “Yes,” you should look for specific things in your audit records. You also need a plan for what records to audit. Your plan should focus on the essential records and any problem areas identified during previous audits. You should always start with a list of the previous problem areas because there should be corrective actions that were implemented, and the effectiveness of corrective actions needs to be verified.

Which records are most valuable when auditing technical files?

I recommend selecting 5-7 records to sample. My choices would be: 1) the ERC checklist, 2) the Declaration of Conformity, 3) labeling, 4) the risk management file, 5) the clinical evaluation report, and 6) post-market surveillance reports, and 7) design verification and validation testing for the most recent design changes. You could argue that my choices are arbitrary, but an auditor can always ask the person they are planning to audit if these records would be the records that the company is most concerned about. If the person has other suggestions, you can change which records your sample. However, you don’t want to sample the same records every year. Try mixing it up each year by dropping the records that looked great the previous year, and adding a few new records to your list this year.

What to look for when auditing technical files

The first thing to look for when you audit records: has the record been updated as required? Some records have a required frequency for updating, while other records only need to be updated when there is a change. If the record is more than three years old, it is probably outdated. For clinical evaluation reports and post-market surveillance reports, the new EU regulations require updating these reports annually for implantable devices. For lower-risk devices, these reports should be updated every other year or once every three years at a minimum.

Design verification and design validation report typically only require revisions when a design change is made, but a device seldom goes three years without a single change–especially devices containing software. However, any EO sterilized product requires re-validation of the EO sterilization process at least once every two years. You also need to consider any process changes, supplier changes, labeling changes, and changes to any applicable harmonized standards.

Finally, if there have been any complaints or adverse events, then the risk management file probably required updates to reflect new information related to the risk analysis.

Which record should you audit first?

The ERC, or Essential Performance and Safety Requirements checklist, is the record you should audit first. First, you should verify that the checklist is organized for the most current regulations. If the general requirements end with section 6a, then the checklist has not been updated from the MDD to the new regulations–which contains nine sections in the general requirements. Second, you should make sure that the harmonized standards listed are the most current versions of standards. Third, you should ensure that the most current verification and validation reports are listed–rather than an obsolete reports.

More auditor training on technical files…

We recorded a live webinar intended to teach internal auditors and consultants how to perform a thorough audit of CE Marking Technical Files against the requirements of the new European regulations–Regulation (EU) 2017/745.

With access to this training webinar, we are also providing a native presentation slide deck, and an audit report template, including checklist items for each of the requirements in Annex I, II, III, and IV of the MDR.

We also provide an exam (i.e., a 10-question quiz) to verify training effectiveness for internal auditors performing technical file auditing. If you submit the completed exam to us by email in the native MS Word format, we will correct the exam and email you a training certificate with your corrected exam. If you have more than one person that requires a training certificate, we charge $49/exam graded–invoiced upon completion of grading.

Technical File Audit Report Auditing Technical Files

Technical File Auditing for Compliance with MDR

This webinar provides an audit report template and teaches auditors how to conduct technical file auditing for compliance with Regulation (EU) 2017/745.

Price: $129.00

In addition to this webinar on auditing technical files, other training webinars are available. For example, we have a webinar on risk management training. If your firm is preparing for compliance with the new MDR, you might also be interested in the following information provided on this website:

A CE Marking procedure including templates for the Essential Requirements, a Declaration of Conformity, and a Technical File Index.
A blog outlining how to create your Quality Plan for compliance with the MDR
A blog explaining three significant differences between the requirements of a Technical File for CE Marking and a 510(k) submission
An 8-part webinar-based course reviewing the requirements of Regulation (EU) 2017/745 and comparing the requirements with the MDD (93/42/EEC as modified by 2007/47/EC).

Please note: A link for logging into this Zoom webinar will be delivered to the email address provided in the shopping cart transaction. After verifying the transaction, please check your email for the login information. To view the available webinars, click here. If you cannot participate in the live Zoom webinar, a link for downloading the recording will be emailed to you.

Auditing Technical Files Read More »