The incoming inspection process is my favorite process to audit, and it is the best process for teaching new auditors.
The above video demonstrates how to use a turtle diagram to conduct a process audit of the receiving inspection process. However, this article goes into more detail. You will learn what to look at and what to look for in each part of the audit process.
Opening meetings are not required for first-party (i.e., internal) and second-party (i.e., supplier) audits. Only third-party auditors are required to have a formal opening meeting. Having an opening meeting is always a good idea, but keep it brief and use a checklist. Try to set the tone for the audit with your opening meeting. This will be your second impression because you already had a conversation with the process owner in preparation for the meeting. However, you want to give everyone present for the opening meeting that you exhibit all the personality characteristics of a good auditor as defined by ISO 19011:2018. Professionalism, organization, and integrity should be obvious to everyone in the room. However, don’t forget to smile and be polite because your auditee might be very nervous. FDA inspectors seem to have an unwritten rule book (i.e., in addition to QSIT) that encourages them to intimidate the companies they inspect.
Step 1 – “Briefly, please describe the incoming inspection process.”
The purpose of this section is not to duplicate the level of detail found in the procedure. It is meant to provide a brief description of the process. Ideally, you want to write a single sentence for the incoming inspection process’s what, where, when, who, and how. A maximum of five sentences is needed to answer those five questions. The process owner should provide the description, and there is no need for them to go into extreme detail because you have at least six more questions to ask (see steps 2-7 below). If you are doing a supplier audit or an audit of a company you don’t work for, you might want to have a few “ice breaker” questions that precede this question. For example, you might ask the person’s name, title, and the number of years they have worked for the company. You might also consider stealing my favorite auditor disclaimer, “If you see me writing furiously, don’t worry. I’m required to write down objective evidence supporting conformity with requirements. If I start asking the same question three different ways, and I’m not writing any notes, that means I am having trouble finding evidence of conformity, and I need your help.”
Step 2 – “What are the inputs that trigger incoming inspection?”
Inputs and outputs of any process refer to both information and physical items. For 100% administrative processes, you may not have any physical items. Incoming inspection, however, has physical goods you receive from suppliers and inspecting. Therefore, the process inputs you are looking for are physical goods and quality system records associated with those goods. For example, if a bunch of titanium round bars were ordered by a buyer in your purchasing department, the physical goods are the titanium bars. The purchase order is one of the quality system records. Other input records that are usually requested to be shipped with the titanium include a packing slip, a certification of analysis, and a dimensional inspection report. It is common to see the incoming inspection activity be delayed because the records are not included with the shipment from the supplier. One recommendation for a process improvement is to require the supplier to send records electronically at the time of shipment instead of sending hardcopies with the product. Statistical inspection sampling plans and work instructions are often confused with input records. These documents are needed to start the incoming inspection, but these are documents that belong in step six of the turtle diagram.
Step 3 – “What are the outputs of the incoming inspection process?”
After incoming inspection is completed there is a requirement to identify the status of the physical product (i.e., accepted or rejected). Usually, a green tag will be used to identify the product as accepted. The tag will also identify the part number, lot, and quantity of product accepted. If the product is titanium, each bar will get a tag. The product will then be transferred to a designated storage area. If you are conducting an audit of a supplier, or a full quality system audit, auditing the warehouse for storage and handling processes is a logical next process. The auditor should look for whether product is segregated in designated locations for specific types of product or if the storage locations are “random” but identified electronically in a material resource planning (MRP) system. The quality system records output from the incoming inspection process will be inspection records and either a green release tag or red rejection tag. If the product is rejected, the product shall be transferred to a quarantine area for nonconforming product and a nonconforming material record (i.e., NCMR) is initiated. Therefore, the process for controlling nonconforming material is another process that could be a logical next process to audit.
Step 4 – “What resources are needed for this process?”
This part of the process approach to auditing is one of the most neglected parts of the quality system. Resources include the facility infrastructure, manufacturing equipment, measurement devices used for inspection, and quality system software used to maintain records of incoming inspection. In this part of the process audit the auditor must be observant. Maintenance records might be located on the side of equipment and they can be reviewed as the auditor walks through the area. This would be an opportunity to interview personnel to make sure they can explain the maintenance process and the equipment maintenance is being performed as planned. The auditor should also determine if equipment validation is required. If the equipment is automated (e.g., automated optical inspection), then an installation qualification (i.e., IQ) should be requested as a quality system record to review at the end of the process or as part of the process for process validation. If the inspection area includes a metrology lab, then the environment may be temperature and humidity controlled. In these types of environments, records of environmental monitoring and trending of environmental conditions should be verified. Lighting, magnification, and particulate filtration could be other environmental requirements for the inspection area. Pest control should be verified in the receiving area, inspection area, and storage areas. The receiving area and warehouse storage are common areas to find pests. Calibration identification should be recorded as a potential follow-up trail for any measurement devices used in the inspection area, and if software is used you will want to verify that quality system software tool validation has been performed.
Step 5 – “Who performs this process?”
A combination of three different roles and responsibilities are typical for this process: 1) department manager, 2) receiving personnel, and 3) inspection personnel. Sometimes one or more of these roles will be combined into one job. The activities sometimes are only performed for a few hours each day, and the personnel that perform the incoming inspection process are assigned to other roles, such as warehouse storage, handling, and shipping. Auditors should always try to interview one or more of the people doing the receiving and inspection activities instead of limiting the interviews to the process owner. Often I will ask the personnel to demonstrate the receiving process and the inspection process. In order to make sure this is possible, you will need to communicate that you want to observe these activities prior to the audit or during the opening meeting. If you don’t, the receiving and inspection activities may already be completed before you start to interview the personnel. Any personnel that are unable to explain the tasks they perform may be targets for verification of training records, effectiveness of training, and competency.
Step 6 – “How is this process performed?”
If an auditor interviews personnel, most people will describe the process in a very haphazard way and steps will be missed. This is why asking people to demonstrate the process is better. The best method is for the person to access the current, approved work instruction or procedure for the process. Then the person should follow the work instruction step-by-step. This allows the person to use the work instruction or procedure as a “crutch” and reduces their nervousness. This also eliminates the skipping steps if the procedures and work instructions are sufficiently detailed. Any blank forms used and statistical inspection standards are also considered quality system documents that define how the process is performed. Sometimes the process owner will provide these documents during their interview, and other times this documents are provided as audit preparation documents. If the documents are not provided in advance the auditor should make sure that they review the documents during observation of activities being performed. This is where an auditor may identify the use of obsolete quality documents, missing details in the documents, and details that are inconsistently followed by personnel.
Step 7 – “What metrics are important for this process?”
Whenever I ask, “What metrics are important in this process?” I typically get a blank stare. Hundreds of business management leaders subscribe to the concept of “what gets managed gets done.” You are also required to establish metrics for your quality system processes in accordance with Clause 8.2.5. Therefore, you need to establish at least one metric, if not more than one. Auditing can help identify opportunities for improvement (OFI), but metrics are the best source of OFIs for a quality system.
Do you need a closing meeting?
You should always conduct a closing meeting for your audits. However, it is also a best practice to summarize your findings for the process owner before you move on to the next process. If some records remain to be reviewed, ensure the process owner knows that the audit results are pending an outcome of reviewing the remaining records. Consider adopting the “sandwich” approach to presenting your findings: 1) something positive, 2) any nonconformities, and 3) something positive. The approach sandwiches the “bad news” between two pieces of “good news.” If you are working as part of a team, the lead auditor should always be aware of the results of your audit. The manager responsible for the process (i.e., the process owner) should also be aware of the results. Do everything you can to prevent unpleasant surprises at the end of the audit.
When you describe any nonconformities, make sure that you include all of the following information:
the grading of the finding (i.e., MDSAP scoring or Major/Minor)
a single sentence stating the finding
the requirement, including a reference to the applicable regulation or standard
objective evidence from your notes
Whenever possible, email a draft of the wording for your nonconformities to the process owner so they can be prepared with clarification questions during the closing meeting. Make sure you agree with your lead auditor before sending the wording of the finding, and copy them on the email communication. If the process owner has initiated immediate corrective action(s), make sure you note this in your report.
Finalizing your audit report
If you are conducting a supplier audit, you need to give the supplier formal feedback from the audit. You will need an audit report for your quality system records, but you are not required to give the supplier the full report. You might provide a summary of the audit for the supplier instead. If you do this, you should include a copy of that communication in your quality system record (e.g., an appendix to your audit report). If you are going to provide a summary of findings, the content should include at least the following:
Everyone has a favorite resource they use to answer regulatory questions, but can you trust OpenAI or Elsmar to answer correctly?
If you are deathly afraid of trying new technology, the image above is a screen capture from OpenAI describing “itself.” OpenAI is artificial intelligence (AI), but it is not self-aware yet. The image below is a screen capture from the “About” webpage for Elsmar Cove. This article was the oldest post on the Medical Device Academy, and it described how to use Elsmar Cove as a resource for quality systems and regulatory questions. To update that blog, we are comparing the use of OpenAI with Elsmar Cove. Just in case you were wondering, Elsmar Cove is #6 on our list of favorite search tools, and OpenAI is #5:
Search the FDA.gov device databases – 3rd place, you can email DICE or call 301.796.7100
Search Google (world’s #1 search engine) – 4th place, ask any 5-year-old for help
Are the answers provided by OpenAI and Elsmar Cove accurate?
To test the accuracy of a common regulatory question, we chose a question we weren’t 100% sure about when a client asked last month. I asked my team, but nobody was 100% certain. Basil Systems is limited to submission and post-market surveillance data. I searched FDA.gov, but it was not clear. Google gave us a link to the FDA website. I asked a couple of ex-FDA consultants, but they gave me outdated information. On Thursday, March 30, 2023, I asked Lisa King during an AAMI course I was co-teaching. Lisa is a Consumer Safety Officer at the FDA responsible for reviewing device entries into the FDA. She is also in very high demand for public training courses. She said the contract manufacturers used to be exempt from registration if they shipped to a legal manufacturer first. The regulations changed, and now 100% of contract manufacturers making a finished device must register with the FDA. She also clarified that the FDA doesn’t use the term “legal manufacturer.”
As you can see from the above answer provided by OpenAI, the ChatGPT engine [i.e., Model: Default (GPT-3.5)] effectively produces the correct answer. Using the same wording for the regulatory question, “Does a foreign contract manufacturer need to register with the FDA if they are shipping the medical device to the legal manufacturer first before the device is exported to the USA?” there were no results from Elsmar Cove. After several attempts, I found what I was looking for using the following search terms, “FDA registration of contract manufacturers.” There were multiple related search results, but the most useful discussion threads in the Elsmar Cove discussion forum were:
The most succinct correct answer in the forum is copied below.
Can you trust OpenAI and Elsmar Cove to answer your regulatory questions?
OpenAI is only as effective as the data used to train it. This is constantly evolving, but we have identified search results that were 100% accurate, results that were outdated, and results that were scary wrong. The same is true of discussion forums. Elsmar Cove is one of the best discussion forums for the medical device industry, but people also use ASQ, RAPS, and AAMI. The quality of the information provided depends upon the knowledge and experience of the people participating in the forum, but it also depends upon the forum’s moderation. Elsmar Cove has some experienced moderators with decades of experience. There is always the chance that the most experienced person in the world could answer your regulatory question incorrectly. This usually creates a problem because everyone else in the forum hesitates to challenge a recognized expert. Therefore, regardless of which resource(s) you use, always try to get a reference to the trustworthy source of the applicable regulation. Even Lisa King could make a mistake, but she immediately said, you can find the regulations in the US Code of Federal Regulations (i.e., 21 CFR 807). The bottom line is, always do your fact-checking and reference your source(s).
Would you like to learn nine ways to improve your quality system procedures? One method is precisely the opposite of our advice from 2011.
During a CAPA course I taught on Friday, January 28, 2011, one of the attendees asked if we teach a course on “How to write better quality system procedures.” Unfortunately, we could only offer material from a course about “Training the trainer.” That “Training the trainer” course focused on visual communication. Several books related to Lean Manufacturing explain how to use visual communication to replace text (i.e., “a picture says a thousand words”). During my ride home, however, I thought of a few other ideas that might help anyone writing or re-writing a procedure. The article was updated and posted as a new blog on Tuesday, March 28, 2023.
1. Use a standardized template for your procedures
In 2013 we published a blog about using a procedure template where we described our 12-part procedure template (i.e., TMP-001). You don’t have to mimic our template, but using a template will accelerate the speed of your writing when you create procedures, and it makes sure you don’t forget any of the essential elements. In addition, using templates ensures a consistent format that makes it easier for everyone to find the information they are looking for. Just make sure that your document control procedure allows flexibility to deviate from the template. The ISO 13485:2016 standard does require a “mandatory” format. Referring to your template as “suggested formatting” will avoid unnecessary nonconformities.
2. Create a process “turtle diagram” for each quality procedure
All of the procedures that Medical Device Academy created have a flow chart at the beginning of the procedure showing the procedures and forms associated with processes that are inputs to that procedure and outputs from that procedure. To systematically improve our procedures, we will be systematically replacing those flow charts with turtle diagrams for each process. This will give more detail than our current flow charts, and internal and external auditors can use the turtle diagrams to understand process interactions.
3. Avoid making unnecessary references to regulations and standards
If you are writing a procedure on risk management—it makes sense to reference ISO 14971. It does not make sense to reference all the other risk analysis standards unless you specifically use them to perform risk analysis. ISO 14971:2019, Clause 4.1, also states that you “shall establish, implement, document, and maintain an ongoing process for” risk management activities. However, the ISO 14971 standard is not directly linked to other procedures. Therefore, ISO 14971 should only be referenced in another if you are using it in that procedure or referencing it directly. For example, the Quality Manual (i.e., POL-001) explicitly references ISO 14971. In contrast, the design control procedure (i.e., SYS-008) references the risk management procedure (SYS-010) but doesn’t reference ISO 14971.
Concerning regulations, you should only reference regulations if the procedure meets a specific requirement. Color coding with symbols should demonstrate traceability to requirements (see method #5 below for further explanation). Rather than adding a reference to regulations in a procedure where there is no requirement, a better approach is to indicate in the Quality Manual that only procedures that have specific requirements will reference the regulations, such as 21 CFR 820 or Part 1 of the Canadian MDR.
4. Track standards, regulations, and the version used in your procedures
In the original 2011 version of this article, we advised quality managers to “avoid including the revision of a standard” because “this is just another opportunity for unnecessary nonconformities.” However, we find that our team has trouble identifying every procedure that a change in regulation or a standard might impact. A systematic process is needed to identify every procedure referencing a regulation or standard. Therefore, we will reference all impacted procedures next to the regulation or standard in our Master Document List (i.e., LST-001). References to the regulations will be added to the main tab for policies, procedures, and work instructions (i.e., [POL, SYS, and WI]). References to the standards will be added to the tab for documents of external origin (i.e., [Doc Ext Origin]).
Many people feel that you should not reference the version of a standard in a procedure because adding the version of the standard increases the number of documents that need to be updated when a standard changes. However, if you are only referencing standards in procedures when it is necessary, then that procedure should be reviewed and updated for the need to be changed. Updating the version of the Standard referenced is the best way to document that a gap analysis against the new version has been completed and the necessary updates were made to the procedure.
5. Use color coding and symbols in your quality system procedures
Matthew Walker, Medical Device Academy’s manager of the human factors team, has systematically updated many of our procedures to the EU Medical Device Regulations 2017/745 and the In Vitro Diagnostic Regulations 2017/746. When he updates our procedures, he references the regulations and applicable ISO 13485:2016 clauses. During certification audits, certification body auditors sometimes have difficulty finding where specific requirements are located in the procedures. Therefore, Matthew added color-coded clause references for our clients and auditors as a corrective action. To make the procedures inclusive for people that are color-blind, Matthew added symbols to supplement the color coding. The extra addition of symbols has proven invaluable because now anyone can search the documents electronically for a symbol to find where all the references are located.
6. Indicate the process owner and training requirements associated with each procedure
Identifying the process owner and training requirements in every procedure makes it easier to define who is responsible for reviewing and revising procedures. For the training requirements, the process owner should specify who needs to be trained on the process. Why? They know the procedure best. If there is a “grey area,” this should be resolved with the department manager for the job function. In addition, retraining requirements should be specified. The training section should also clarify if retraining is required when revising a procedure. If the revision is minor, training should only be necessary for people not trained on a previous revision.
7. Adopt thePlan-Do-Check-Act (PDCA) model for the structure of quality system procedures
For the “Plan” portion, the procedure should explain how to prepare to do something. This planning activity can apply to anything from planning to perform an audit to planning to inspect incoming raw materials. The “Do” portion is what most people refer to as the “Procedure” section. The “Check” portion of the procedure is a great place to specify the monitoring and measurement requirements for the process (see Section 8.1 of the Standard). Finally, the “Act” portion of the procedure should indicate what to do when target metrics are unmet. For example, what should be done when an alert limit is reached? What should be done when an action limit is reached?
8. Include therevision history of quality system procedures
It’s helpful to know which Document Change Notice (DCN) approved the document revision, why the changes were made, the nature of the changes, whether there is a related corrective action, and when the change was made. This will also tell auditors whether there is anything new to audit since the previous internal or external audit. This section is usually near the beginning of our procedures, but it doesn’t matter if the revision history is at the end or the beginning. However, it does help to be consistent.
9. Identify the form number, location, and retention period for each record
We have a section about quality system records near the end of every procedure. This section lists each quality system record that is associated with the procedure. The relevant form is referenced, but we recommend storing these records in electronic or paper folders labeled with the form number. If the files are digital, a hyperlink should be included. If the files are paper, then you should list the physical location of storage. The retention period can be listed in each procedure. Still, it will be essential to ensure that this information matches the regulatory requirements and record retention requirements in your “Control of Records” procedure (i.e., SYS-002).
Read this article to learn why ISO 19011 standard is a vital guidance for anyone that audits quality systems or manages an audit program.
What is ISO 19011?
ISO 19011 is a seven-part international standard for auditing management systems. The standard defines the eight principles of auditing (e.g., the process approach to auditing), provides guidance on managing audit programs and conducting audits, and includes recommendations for evaluating people for competency. There is also an appendix with details on conducting on-site and remote audits.
If you have ever taken a lead auditor course forISO 13485, or one of the other quality management system standards, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Quality Management Systems.” In 2018, ISO 19011 was updated, and the changes were not superficial. If you need to purchase a copy of ISO 19011:2018, the Estonian Center for Standardization and Accreditation is the least expensive source we know.
ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting internal and external audits, and determining auditor competency. One of the most common points of confusion in the lead auditor course is the difference between first, second, and third-party audits. In the first edition of this Standard, the difference between first, second, and third-party audits was just a note at the bottom of page one and the top of page two. The note was also not clear. In the second edition of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear. Table 1 was modified further in the 3rd edition to include a bottom row that remains unchanged in the 3rd edition, released in 2018.
Figure 1, found in Clause 5.1 of the 2nd edition, was combined with Figure 2, found in Clause 6.1 of the 2nd edition. The combined figure is now Figure 1 in the 3rd edition. The combined scope of Figure 1 is now a “Process flow for the management of an audit program” and a “Process flow for conducting an audit.” The figure categorizes the various stages of audit program management and conducting an audit into the Plan-Do-Check-Act (PDCA) cycle. We highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard.
The 2018 version still includes an opening meeting checklist (i.e., Clause 6.4.3) and a closing meeting checklist (i.e., Clause 6.4.10). Figure 3 in the 2nd edition, “Overview of the process of collecting and verifying information,” was a poor example of a flow chart. The committee did not update the figure when the standard was updated for the 3rd edition. Therefore, we updated the figure below to provide additional traceability to the Clauses of the Standard. If you incorporate this figure into your quality auditing procedure, you should substitute references to your procedure’s sections instead of the clauses of the standard.
Competency Requirements in ISO 19011
Many audit procedures neglect to define the qualifications and methods for determining thecompetency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures we read include qualifications for a “Lead Auditor,” but we seldom see anything regarding competency. Unfortunately, this Standard only explicitly addresses the “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When we teach people how to be Lead Auditors, we spend more than an hour on this topic alone.
The Standard would be more effective by providing an example of how third-party auditors become qualified as a Lead Auditor. Third-party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meetings, conducting the audit, closing meetings, and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e., – Stage 2 certification or re-certification), and another qualified lead auditor must evaluate you and provide feedback.
Appendices in ISO 19011
The appendices were the last significant additions to this Standard in 2011 (i.e., 2nd edition). Annex A provided examples of discipline-specific knowledge and skills of auditors. This section was eliminated from the 3rd edition of ISO 19011:
“Due to the large number of individual management system standards, it would not be practical to include competence requirements for all disciplines.” – Copied from the Foreward
I think providing adding a short Annex to each management system standard that defines recommended discipline-specific knowledge would be helpful. Still, that kind of change would need to be initiated with the next version of ISO 9001.
Appendix B in the 2nd edition is now Appendix A in the 3rd edition of ISO 19011. A table (Table A.1 – Audit Methods) compares conducting on-site and remote audits. We were pleased to see that conducting interviews is a significant part of remote auditing in this table. Section A.17 in the appendix provides suggestions for conducting interviews. Still, if you exhibit all 13 professional behavior traits found in Clause 7.2.2, you don’t need advice on speaking with people. For the rest of us mortals, we could use a five-day course on interviewing alone. To improve your skills in this area, ask an experienced auditor with solid interviewing skills to watch and comment on a recording of a virtual audit you perform. Watching yourself audit is cringe-worthy, but we guarantee you will improve.
What are the primary changes to the 2018 version of the standard?
There are seven main differences between the second edition, published in 2011, and the third edition of ISO 19011, released in 2018:
addition of a seventh principle of auditing in sub-clause 4(g) (i.e., risk-based approach);
more guidance on audit program management in Clause 5, including audit program risk;
expansion of Clause 6 on conducting an audit–especially Clause 6.3 on audit planning;
expansion of auditor competence requirements in Clause 7;
updating of terminology to emphasize processes rather than objects;
removal of an annex containing competence requirements for specific quality management systems;
expansion of Annex A to include guidance on new auditing concepts such as remote audits.
Risk-based auditing is the most significant change in the 2018 version of ISO 19011
One of the main differences between ISO 19011:2018 and the previous 2011 version is the addition of a “risk-based approach” to the principles of auditing. Specifically, clause 4(g) of the guidelines for auditing management systems is, “The risk-based approach should substantively influence the planning, conducting and reporting of audits to ensure that audits are focused on matters that are significant for the audit client, and for achieving the audit program objectives.” A lot of people are unsure of what is meant by a risk-based approach. Still, the key to understanding this is to focus on the definition of risk. From a product perspective, the risk is the “combination of the probability of occurrence of harm and the severity of that harm.” From a process perspective, the risk is the “effect of uncertainty on an expected result” (ISO 9001:2015, clause 3.09). Therefore, auditors should emphasize medical devices with the highest severity of harm and devices with a high probability of hazards or hazardous situations. When an auditor focuses on a process rather than a specific medical device, auditors should emphasize any processes that are not under control and any recent process changes.
What is risk-based auditing?
Risk-based auditing considers the risks of failing to achieve audit objectives and the opportunities created by choosing various audit methods and strategies. For example, a desktop audit of procedures might be appropriate if you are conducting your first internal audit for a new quality system. Alternatively, a desktop audit would be a waste of time if you are auditing a mature quality system where very few changes to procedures have been made in the past year. Using the element approach to auditing is unlikely to add much value. Audits are meant to be a sampling. Therefore, you should focus on areas of importance where previous nonconformities were identified, any new products or processes, and anything that changed significantly.
Auditor selection should also be risk-based
Suppose you are conducting a supplier audit as part of your initial supplier qualification for a critical component supplier or contract manufacturer. In that case, you should consider doing a team audit with a multi-disciplinary team. This is a risk-based approach to the supplier qualification process, which ensures that subject matter experts evaluate each process instead of auditors with a general quality assurance background. This approach also forces more of your personnel to introduce themselves to the new supplier, and the audit will develop more reliable communication channels between your two companies. Alternatively, if you are conducting a routine internal audit of a production process, you might select a new lead auditor to conduct the audit. You don’t expect any significant findings in a routine internal audit of an established production process. In your role as an audit program manager, you need to match the new lead auditor to a process that will force them to look at all aspects of the process approach to auditing. Specifically, process validation, calibration, maintenance, and process monitoring may not apply to other administrative process areas, such as purchasing.
Risk-based auditing should influence your auditing schedule
The frequency of auditing suppliers and internal process areas should reflect the associated risks. Therefore, when you create or update your auditing schedule, you should consider the risk level of the products being audited and the process being audited. Production processes with a moderate or high level of non-conforming products may need to be audited more than once yearly. Still, a supplier with an excellent track record of extremely high quality and on-time delivery may be audited in alternating years. If you previously scheduled a remote audit, you may want to alternate to conducting an on-site audit the next time.
The duration of your audits should not always be the same either. Suppose one production process makes one product in low volume, and another production process makes multiple products in high volume. In that case, you should not schedule a two-hour internal audit for both processes every year. The low-volume production process may only need a one-hour audit once per year. In contrast, the high-volume process may require a four-hour internal audit or multiple annual audits.
Risk-based auditing applied to remote supplier auditing
The risk-based auditing approach was added to ISO 19011:2018 as the seventh principle of auditing. This represents the most significant change to that standard, but how does it apply to remote auditing? Despite the opportunities created by remote auditing, there are also risks associated with auditing suppliers remotely. People worry about auditees hiding hazardous situations or unacceptable environmental conditions such as filth or disrepair. However, unacceptable cleanliness and maintenance practices don’t happen overnight. Therefore, you should expect a clean and well-maintained facility to remain that way. One approach is to alternate between remote and on-site audits to verify the overall condition of a supplier’s facility. Therefore, the risk of auditees hiding objective evidence is more an issue of trust than a highly probable occurrence.
The more probable risks associated with remote auditing are related to the potential lack of availability of records. This is especially important for paper-based quality systems. Most people try to address this risk by scanning paper documents and records, but scanning documents have limited value. Scanning paper documents is more efficiently performed in a large batch by an automated or semi-automated process. Also, auditors and inspectors typically focus on the most recent records, and auditors and inspectors rarely sample 100% of the records. Therefore, the best risk controls include the following:
Ask a guide to send a digital picture of the record.
Use a tripod-mounted HD webcam focused on a music stand or similar surface.
Ask the auditee to read the document while you take notes.
In our experience, you will probably rely on all three risk controls, but it is unlikely to delay the audit. However, in response to the limited physical access to medical device facilities and personnel, certification bodies are sending out questionnaires to assess the risk of being unable to achieve audit objectives or cover the required scope of surveillance and recertification audits. As the audit program manager, you can reduce these risks by working with supply chain managers to develop new supplier questionnaires that specifically ask questions about the capability of supporting audits remotely. In particular, it would be essential to obtain facility maps to identify areas with inadequate cellular coverage and identify records that are only available in hardcopy format.
The FDA patches the regulations with guidance documents, but there is a desperate need to modernize 21 CFR 820 to ISO 13485.
FDA Proposed Amendment to 21 CFR 820
On February 23, 2022, the FDA published a proposed rule for medical device quality system regulation amendments. The FDA planned to implement amended regulations within 12 months, but the consensus of the device industry is that a transition of several years would be necessary. In the proposed rule, the FDA justifies the need for amended regulations based on the “redundancy of effort to comply with two substantially similar requirements,” creating inefficiencies. In public presentations, the FDA supporting arguments for the proposed quality system rule change relies heavily upon comparing similarities between 21 CFR 820 and ISO 13485. However, the comparison table provided is quite vague (see the table from page 2 of FDA’s presentation reproduced below). The FDA also provided estimates of projected cost savings resulting from the proposed rule. What is completely absent from the discussion of the proposed rule is any mention of the need to modernize 21 CFR 820.
Are the requirements “substantively similar”?
The above table provided by the FDA claims that the requirements of 21 CFR 820 are substantively similar to the requirements of ISO 13485. However, there are some some aspects of ISO 13485 that will modernize 21 CFR 820. The areas of impact are: 1) software, 2) risk management, 3) human factors or usability engineering, and 4) post-market surveillance. The paragraphs below identify the applicable clauses of ISO 13485 where each of the four areas are covered.
Modernize 21 CFR 820 to include software and software security
Despite the limited proliferation of software in medical devices during the 1990s, 21 CFR 820 includes seven references to software. However there are some Clauses of ISO 13485 that reference software that are not covered in the QSR. Modernizing 21 CFR 820 to reference ISO 13485 will incorporate these additional areas of applicability. Clause 4.1.6 includes a requirement for validation of quality system software. Clause 7.6 includes a requirement for validation of software used to manage calibrated devices used for monitoring and measurement. Clause 7.3 includes a requirement for validation of software embedded in devices, but that requirement was already included in 21 CFR 820.30. The FDA can modernize 21 CFR 820 further by defining Software as a Medical Device (SaMD), referencing IEC 62304 for management of the software development lifecycle, referencing IEC/TR 80002-1 for hazard analysis of software, referencing AAMI TIR57 for cybersecurity, and referencing ISO 27001 for network security. Currently the FDA strategy is to implement guidance documents for cybersecurity and software validation requirements, but ISO 13485 only references IEC 62304. Then only aspect of 21 CFR 820 that appears to be adequate with regard to software is validation of software used for automation in 21 CFR 820.75. This requirement is similar to Clause 7.5.6 (i.e., validation of processes for production and service provisions).
Does 21 CFR 820 adequately cover risk management?
The FDA already recognizes ISO 14971:2019 as the standard for risk management of medical devices. However, risk is only mentioned once in 21 CFR 820. In order to modernize 21 CFR 820, it will be necessary for the FDA to identify how risk should be integrated throughout the quality system requirements. The FDA recently conducted two webinars related to risk management of medical devices, but implementing a risk-based approach to quality systems is a struggle for companies that already have ISO 13485 certification. Therefore, a guidance document with examples of how to implement a risk-based approach to quality system implementation would be very helpful to the medical device industry.
Modernize 21 CFR 820 to include Human Factors and Usability Engineering
ISO 13485 references IEC 62366-1 as the applicable standard for usability engineering requirements, but there is no similar requirement found in 21 CFR 820. Therefore, human factors is an area where 21 CFR 820 needs to be modernized. The FDA has released guidance documents for the human factors content to be included in a 510k pre-market notification, but the guidance was released in 2016 and the guidance does not reflect the FDA’s current thoughts on human factors / usability engineering best practices. The FDA recently released a draft guidance for the format and content of human factors testing in a pre-market 510k submission, but that document is not a final guidance document and there is no mention of human factors, usability engineering, or even use errors in 21 CFR 820. Device manufacturers should be creating work instructions for use-related risk analysis (URRA) and fault-tree analysis to estimate the risks associated with use errors as identified the draft guidance. These work instructions will also need to be linked with the design and development process and the post-market surveillance process.
Modernize 21 CFR 820 to include Post-Market Surveillance
ISO/TR 20416:2020 is a new standard specific to post-market surveillance, but it is not recognized by the FDA. There is also no section of 21 CFR 820 that includes a post-market surveillance requirement. The FDA QSR focuses on reactive elements such as:
21 CFR 820.100 – CAPA
21 CFR 820.198 – Complaint Handling
21 CFR 803 – Medical Device Reporting
21 CFR 820.200 – Servicing
21 CFR 820.250 – Statistical Techniques
The FDA does occasionally require 522 Post-Market Surveillance Studies for devices that demonstrate risks that require post-market safety studies. In addition, most Class 3 devices are required to conduct post-approval studies (PAS). For Class 3 devices, the FDA requires the submitter provide a plan for a post-market study. Once the study plan is accepted by the FDA, the manufacturer must report on the progress of the study. Upon completion of the study, most manufacturers are not required to continue PMS.
How will the FDA enforce compliance with ISO 13485?
It is not clear how the FDA would enforce compliance with Clause 8.2.1 in ISO 13485, because there is no substantively equivalent requirement in the current 21 CFR 820 regulations. The QSR is 26 years old, and the regulation does not mention cybersecurity, human factors, or post-market surveillance. Risk is only mentioned once by the regulation, and software is only mentioned seven times. The FDA has “patched” the regulations through guidance documents, but there is a desperate need for new regulations that include critical elements. The transition of quality system requirements for the USA from 21 CFR 820 to ISO 13485:2016 will force regulators to establish policies for compliance with all of the quality system elements that are not in 21 CFR 820.
Companies that do not already have ISO 13485 certification should be proactive by 1) updating their quality system to comply with the ISO 13485 standard and 2) adopting the best practices outlined in the following related standards:
AAMI/TIR57:2016 – Principles For Medical Device Security – Risk Management
IEC 62366-1:2015 – Medical devices — Part 1: Application of usability engineering to medical devices
ISO/TR 20416:2020 – Medical devices — Post-market surveillance for manufacturers
ISO 14971:2019 – Medical Devices – Application Of Risk Management To Medical Devices
IEC 62304:2015 – Medical Device Software – Software Life Cycle Processes
ISO/TR 80002-1:2009 – Medical device software — Part 1: Guidance on the application of ISO 14971 to medical device software
ISO/TR 80002-2:2017 – Medical device software — Part 2: Validation of software for medical device quality systems
What is the potential impact of the US FDA requiring software, risk management, cybersecurity, human factors, and post-market surveillance as part of a medical device company’s quality system?
Learn how to become ISO 13485 certified while avoiding the stress that tortures other quality system managers.
Your cart is empty
What is ISO 13485?
ISO 13485 is an international standard for quality management systems that is specific to the medical device industry. ISO 13485:2016 is the most recent version of the standard, and it has become the blueprint for medical device company quality systems globally. If your company wants to design, manufacture, or distribute medical devices you should consider becoming ISO 13485 certified.
Yes, you need to maintain a copy of the ISO 13485 standard as a “document of external origin.” This is needed for reference when you are making updates to procedures in your quality system. If you are looking for the best place to purchase a copy of the ISO 13485:2016 standard, we recommend the Estonian Centre for Standardisation and Accreditation. If you purchase a copy, we recommend selecting the option for a multi-user license so the standard can be used by more than one person in your company and printed. The only difference between the EN ISO version and the International ISO version is that the EN ISO version includes harmonization Annex ZA for compliance with the EU MDR and Annex ZB for compliance with the EU IVDR. This version is also referred to as A11:2021. Here’s a copy of the text from the beginning of the Standard:
“This Estonian standard EVS-EN ISO 13485:2016/A11:2021 consists of the English text of the European standard EN ISO 13485:2016/A11:2021. This standard has been endorsed with a notification published in the official bulletin of the Estonian Centre for Standardisation and Accreditation. Date of Availability of the European standard is 08.09.2021. The standard is available from the Estonian Centre for Standardisation and Accreditation.”
Medical Device Academy’s experience with ISO 13485 training
Rob Packard created his first quality system in the Spring of 2004. In October of 2009, after successfully managing quality systems for three different medical device manufacturers, Rob joined BSI as a Lead Auditor and instructor. In April of 2010, he purchased the 13485cert.com URL and he began to help companies implement quality systems as a consultant (while continuing to audit and train 140 days per year for BSI). In 2011 his medical device blog postings began as a way to help medical device companies. In 2012, Rob began building a library of quality system procedures for a turn-key quality system and selling the procedures from the Medical Device Academy website. Dozens and dozens of consulting clients have successfully achieved ISO 13485 certification with Medical Device Academy’s turnkey quality system procedures, and hundreds of quality systems were audited and/or improved. This ISO 13485 training webinar is also included as part of our turnkey quality system.
Projected Changes for 2023
On February 23, 2022, the FDA published a proposed rule for medical device quality system regulation amendments. The FDA planned to implement amended regulations within 12 months, but the consensus of the device industry is that a transition of several years would be necessary. In the proposed rule, the FDA justifies the need for amended regulations based on the “redundancy of effort to comply with two substantially similar requirements,” creating inefficiencies. The FDA also provided estimates of projected cost savings resulting from the proposed rule. What is completely absent from the proposed rule is any mention of the need for modernization of device regulations.
The QSR is 26 years old, and the regulation does not mention cybersecurity, human factors, or post-market surveillance. Risk is only mentioned once by the regulation, and software is only mentioned seven times. The FDA has “patched” the regulations with guidance documents, but there is a desperate need for new regulations that include critical elements. The FDA has “patched” the regulations through guidance documents, but there is a desperate need for new regulations that include critical elements. The transition of quality system requirements for the USA from 21 CFR 820 to ISO 13485:2016 will force regulators to establish policies for compliance with each of these quality system elements. Companies that do not already have ISO 13485 certification should be proactive by 1) updating their quality system to comply with the standard and 2) adopting the best practices outlined in the following related standards:
AAMI/TIR57:2016 – Principles For Medical Device Security – Risk Management
IEC 62366-1:2015 – Medical devices — Part 1: Application of usability engineering to medical devices
ISO/TR 20416:2020 – Medical devices — Post-market surveillance for manufacturers
ISO 14971:2019 – Medical Devices – Application Of Risk Management To Medical Devices
IEC 62304:2015 – Medical Device Software – Software Life Cycle Processes
ISO/TR 80002-1:2009 – Medical device software — Part 1: Guidance on the application of ISO 14971 to medical device software
ISO/TR 80002-2:2017 – Medical device software — Part 2: Validation of software for medical device quality systems
Previous versions of the ISO 13485 webinars
This 2-part webinar has been previously recorded three different times. Our previous webinar on the 2003 version of ISO 13485 was split into two parts: Stage 1 and Stage 2. That first webinar was recorded in 2015. The webinars were updated in 2016 and again in 2018. We followed the same format, 2-part Stage 1 and Stage 2, for all of the subsequent ISO 13485 training webinars. The Stage 1 webinar focuses on the following processes:
Quality System Documentation
The Stage 2 webinar on the rest of the standard, including but not limited to:
Customer Related Processes
The most recent version of ISO 13485 webinars
The webinars explaining the requirements for ISO 13485 were last updated for 2020. Anyone that purchases these webinars will receive free access to updated versions of the ISO 13485 training webinars. If you are making a new purchase of these two training webinars, the webinars are only being sold as a bundle for $258. You get:
access to the Stage 1 webinar recorded July 24, 2020
access to the Stage 2 webinar recorded July 28, 2020
native slide decks for both webinars
This pair of ISO 13485 training webinars explain precisely what you need to do to implement a quality system compliant with ISO 13485. After you create your own plan (a free template is provided with a subscription), then you can show the recording of these two webinars to your management team so they can implement your plan in the next several months. All deliveries of content will be sent via Aweber emails to confirmed subscribers.
Webinar duration & format
Webinars were hosted live via Zoom in 2020. The Stage 1 webinar was 64 minutes, and the duration of the Stage 2 webinar was 82 minutes. When you purchase this webinar bundle, you will receive a link to download both recorded webinars from our Dropbox folder. In addition, you will receive links to download the native slide deck for each webinar from Dropbox.
Purchase the ISO 13485 training bundle
ISO 13485:2016 Training Webinars – Stage 1 & Stage 2
The webinars explaining the requirements for ISO 13485 were last updated for 2020. Anyone that purchases these webinars will receive free access to updated versions of the ISO 13485 training webinars. If you are making a new purchase of these two training webinars, the webinars are only being sold as a bundle for $258. You get:
1 – access to the Stage 1 webinar recorded July 24, 2020
2 – access to the Stage 2 webinar recorded July 28, 2020
3 – native slide decks for both webinars
Exam and Training Certificate available
Exam – ISO 13485:2016 update
This is a 20 question quiz with multiple choice and fill in the blank questions. The completed quiz is to be submitted by email to Rob Packard as an MS Word document. Rob will provide a corrected exam with explanations for incorrect answers and a training effectiveness certificate for grades of 70% or higher.
There is a big difference between being ISO 13485 certified and being compliant with ISO 13485:2016, the medical devices quality management systems standard. Anyone can claim compliance with the standard. Certification, however, requires that an accredited certification body has followed the requirements of ISO 17021:2015, and they have verified that your quality system is compliant with the standard. To maintain that certification, you must maintain your quality system’s effectiveness and endure both annual surveillance audits and a re-certification audit once every three years.
Step 1 – Planning for ISO 13485 certification
There are six steps in the ISO 13485 certification process, but that does not mean there are only six tasks. The first step in every quality system is planning. Most people refer to the Deming Cycle or Plan-Do-Check-Act (PDCA) Cycle when they describe how to implement a quality system. However, when you are implementing a full quality system, you need to break the “doing” part of the PDCA cycle into many small tasks rather than one big task. You also can’t implement a quality system alone. Quality systems are not the responsibility of the quality manager alone. Implementing a quality system is the responsibility of everyone in top management.
Below you will find seven tasks listed. I did NOT identify these nine tasks as “Steps” in the ISO 13485 certification process, because these tasks are typically repeated for each process in your quality system. Most quality systems are implemented over time, and the scope of the quality system usually grows. Therefore, you are almost certain to have to perform all of the following nine tasks multiple times–even after you receive the initial ISO 13485 certification. As the saying goes, “How do you eat an elephant? One bite at a time.” Therefore, avoid the inevitable heartburn caused by trying to do too much at one time. Implement your quality system one “bite” at a time.
Task 1 – Purchase applicable standards
The first task in implementing an ISO 13485 quality system is to purchase a copy of the ISO 13485:2016 standard, such as the MDSAP Companion Document. You will also need other applicable medical device standards. Some of these standards are general standards that apply to most, if not all, medical devices, such as ISO 14971:2019 for risk management. There are also guidance documents that explain how to use these general standards, such as ISO/TR 24971:2020, and guidance on how to apply ISO 14971:2019. Finally, there are testing standards that identify testing methods and acceptance criteria for things such as biocompatibility and electrical safety. You will need to monitor these standards for new and revised versions. When these standards are updated, you will need to identify the revised standard and develop a plan for addressing the changes.
When you purchase a standard, be sure to buy an electronic version of the standard so you can search the standard for keywords efficiently. You should also consider purchasing a multi-user license for the standard because every manager in your company will need to look up information in the standard. Alternatively, you could buy a paper copy of the standard and locate the standard where everyone in your company can access it. Often I am asked what the difference is between the EN version of the standard and the ISO version of the standard. “EN” is an abbreviation meaning European Standards or “European Norms,” which is based upon the literal translation from the French (i.e., “normes”) and German (i.e. “norm”) languages. “ISO” versions are international standards. In general, the body of the standard is typically identical but harmonized EN standards for medical devices include annexes ZA, ZB, and ZC that identify any deviations from the requirements in three medical device directives (i.e., MDD, AIMD, and IVDD).
Task 2 – Identify which processes are applicable
Clause 1 of ISO 13485 is specific to the scope of a quality system. ISO 9001, the general quality system standard, allows you to “exclude” any clause from your quality system certification. However, ISO 13485 will only allow you to exclude design controls (i.e., clause 7.3). Other clauses within ISO 13485 may be identified as “non-applicable” based upon the nature of your medical device or service. You must also document the reason for non-applicability in your quality manual. Typically, the following clauses are common clauses identified for non-applicability:
Clause 4.1.6 – quality system software
Clause 6.4 – work environment
Clause 7.5.2 – cleanliness of the product
Clause 7.5.3 – installation
Clause 7.5.4 – servicing
Clause 7.5.5 – sterile devices
Clause 7.5.6 – process validation
Clause 7.5.7 – sterilization validation
Clause 126.96.36.199 – implantable devices
Clause 7.5.10 – customer property
Clause 8.3.4 – rework
Task 3 – Assign a process owner to each process
The third task is to assign a process owner to each of the processes in your quality system. Typically, you create a master list of each of the required processes. Usually, the assignments are made to managers in the company who may delegate some or all of a specific process. You should expect most managers to be responsible for more than one process because there are 28 required procedures in ISO 13485:2016, but most companies have fewer than ten people when they first implement a quality system.
Task 4 – Prioritize and schedule the implementation of each process
The fourth task is to identify which processes need to be created first and to schedule the implementation of procedures from first to last. You can and should build flexibility into the schedule, but some procedures are needed at the beginning. For example, you need document control, record control, and training processes to manage all of your other procedures. You also need to implement the following processes to document your Design History File (DHF): 1) design controls, 2) risk management, 3) software development (if applicable), and 4) usability. Therefore, these represent the seven procedures that most companies will implement as early as possible. Procedures such as complaint handling, medical device reporting, and advisory notice procedures are usually reserved for last. These procedures are last because they are not needed until you have a medical device in use.
Task 5 – Create forms, flowcharts, and procedures for each process
Forms create the structure for records in your quality system, and a well-designed form can reduce the need for lengthy explanations in a procedure or work instruction. Therefore, you should consider developing forms first. The form should include all required information that is specified in the applicable standard or regulations, and the cells for that information should be presented in the order that the requirements are listed in the standard. You might even consider numbering the cells of the form to provide an easy cross-reference to the corresponding section of the procedure. Once you create a form, you might consider creating a flowchart next. Flowcharts provide a visual representation of the process. You might consider including numbers in the flow chart that cross-reference to the form as well.
Once you have created a form and a flowchart, you are now ready to write your quality system procedure. Many sections are typically included in a procedure template. It is recommended that you use a template to ensure that none of the basic elements of a procedure are omitted. You might also consider adding two sections that are uncommon to a procedure: 1) risk analysis of the procedure with the identification of risk controls to prevent risks associated with the procedure, and 2) a section for monitoring and measurement of the process to objectively measure the effectiveness of the process. These metrics are the best sources of preventive actions, and some of the metrics might be potential quality objectives to be identified by top management.
Task 6 – Perform a gap analysis of each procedure
Most companies rely upon internal audits to catch missing elements in their procedures. However, audits are intended to be a sampling rather than a 100% comprehensive assessment. Therefore, when a draft procedure is being reviewed and approved for the first time, or a major re-write of a procedure is conducted, a thorough gap analysis should be done before the approval of the draft procedure. Matthew Walker created an article explaining how to conduct a gap analysis of procedures. In addition, Matthew has been gradually adding cross-references to ISO 13485:2016 requirements in each procedure. He is color-coding the cross-referenced clauses in blue font as well. This makes it much easier for auditors to verify that a procedure is compliant with the regulations with minimal effort. The success of these two methods has taught us the importance of conducting a gap analysis of all new procedures.
Task 7 – Train applicable personnel for each process
You are required to document the training requirements for each person or each job in your company. Documentation of training requirements may be in a job description or within a procedure. In addition to defining who should be trained, you also need to identify what type of training should be provided. We recommend recording your training to ensure that new future employees receive the same training. This will ensure consistency. You are also required to maintain records of the training. You must verify that the training was effective, and you need to check whether the person is competent in performing the tasks. This training may require days or weeks to complete. Therefore, you may want to start training people several weeks before your procedure is approved. Alternatively, you can swap the order of tasks and conduct training after the procedure approval. If that approach is taken, then the procedure should indicate the date the procedure becomes effective–typical 30 days after approval to allow time for training.
Task 8 – Approve the procedure
Approval of a procedure may be accomplished by signing and dating the procedure itself, while another approach is to create a document that lists all the procedures and forms being approved at one time. The second method is the method we use in our turn-key quality system. Companies can review and approve as many procedures at one time as they wish. Since this process needs to be defined to ensure that all of the procedures you implement are approved, the document control process is typically the first procedure that companies will approve in a new quality system. The second procedure generally is for the control of records. Then the next procedures implemented will typically be focused on the documentation of design controls: design controls, risk management, usability testing, and software development. The last procedures to be approved are typically complaint handling, medical device reporting, and recalls. These procedures are left for last because you don’t need them until you are selling your medical device.
Task 9 – Start using the procedure and generating records
The last task required for the implementation of a new quality system is to start using the procedures to generate records. All of the procedures will need records before the process can be verified to be effective. Records can be paper-based, or the records can be electronic. Whichever format you use for the record retention needs to be communicated to everyone in the company through your Control of Records procedure and/or within each procedure. If you include the information in each procedure, the records of each procedure should be listed in the procedure, and the location where those records are stored should be identified. Generally, there is no specific minimum number of records to have for a certification audit, but you should have at least a few records for each process that you implement.
Step 2 – Conducting your first internal audit
The purpose of the internal audit is to verify the effectiveness of the quality system and to identify nonconformities before the certification body auditor finds them. To successfully achieve this secondary objective, it is essential to have a more rigorous internal audit than you expect for the certification audit. Therefore, the internal audit should be of equal duration or longer in duration than the certification audit. The internal audit should not consist of a desktop review of procedures. Reviewing procedures should be part of gap analysis (i.e., task 6 above) that is conducted on draft procedures before they are approved. Internal audits should utilize the process approach to auditing, and the auditor should apply a risk-based approach (i.e., focus on those processes that are most likely to contribute to the nonconforming products, result in a complaint, or cause severe injuries and death).
After your internal audit, you will receive an internal audit report from the auditor. You should also expect findings from the internal auditor, and you should expect opportunities for improvement (OFI) to be identified. Experienced auditors can typically identify the root cause of a nonconformity more quickly than most process owners. Therefore, it is recommended for each process owner and subject matter expert to review nonconformities with the auditor and discuss how the nonconformity should be investigated. The root cause must be correctly identified during the CAPA process, and the effectiveness check must be objective to ensure that problems do not recur.
Step 3 – Initiating corrective actions
Corrective actions should be initiated for each internal audit finding immediately, to make sure the findings are corrected and prevented from repeat occurrence before the Stage 1 audit. It will take a minimum of 30 days to implement the most corrective actions. Depending upon the scheduling of the internal audit, there may not be sufficient time to complete the corrective actions. However, you should at least initiate a CAPA for each finding, perform an investigation of the root cause, and begin to implement corrective actions.
Also, to take corrective actions related to internal audit findings, you should look for internal audits from other sources. The diagram below shows several different sources of potential corrective and preventive actions.
Monitoring and measuring each process is the best source of preventive actions, while internal audits are typically the best source of corrective actions. Any quality problems identified during validation are also excellent sources of corrective actions because the validation can be repeated as a method of demonstrating that the corrective actions are effective. However, your ISO 13485 certification auditor will focus on non-conforming products, complaints, and services as the most critical sources of corrective actions. These three sources are prioritized because these three sources have the greatest potential for resulting in a serious injury, death, or recall if corrective actions are not implemented to prevent problems from recurring.
Step 4 – Conducting your first management review
In addition to completing a full quality system audit before your stage 1 audit, you are also expected to complete at least one management review. To make sure that you have inputs for each of the 12 requirements in the ISO 13485:2016 standard, it is recommended to conduct your management review only after you have completed your full quality system audit and initiated some corrective actions. If possible, you should also conduct supplier audits for any contract manufacturers or contract sterilizers. It is recommended to use a template for that management review that is organized in the order of the required inputs to ensure that none of the necessary inputs are skipped. Quality objectives will need to be established long before the management review so that the top management team has sufficient time to gather data regarding each of the quality objectives. Also, you should consider delegating the responsibility for creating the various slides for each input to different members of top management. This will ensure that everyone invited to the meeting is engaged in the process, and it will spread the workload for meeting preparation across multiple people.
At the end of the meeting, top management will need to create a list of action items to be completed before the next management review meeting. Meeting minutes will need to be documented for the meeting, including the list of action items and each of the four required outputs of the management review process. We recommend using the notes section of a presentation slide deck to document the meeting minutes related to each slide. Then the slide deck can be converted into notes pages and saved as a PDF. The PDF notes pages will be your final meeting minutes for the management review. An example of one of these notes pages is provided in the figure below.
One of the more common non-value-added findings by auditors is when an auditor issues a nonconformity because you do not have your next internal audit and your next management review scheduled–even though each may have occurred only a month prior to the Stage 1 audit. Therefore, we recommend that you document your next 12-month cycle for internal audits and schedule your next management review as action items in every management review meeting. The schedule can be adjusted if needed, but this allows top management to emphasize various areas in internal audits that may need improvement. You might even set a quality objective to conduct a minimum of three management reviews per year at the end of your first management review.
Step 5 – Stage 1, Initial ISO 13485 Certification Audit
In 2006, the ISO 17021 Standard was introduced for assessing certification bodies. This is the standard that defines how certification bodies shall go about conducting your initial certification audit, annual surveillance of your quality system, and the re-certification of your quality system. In the past, certification bodies would typically conduct a “desktop” audit of your company before the on-site visit to make sure that you have all the required procedures. However, ISO 17021 requires that certification bodies conduct a Stage 1 audit that assesses the readiness of your company before conducting a Stage 2 audit. Therefore, even if the Stage 1 audit is conducted remotely, the certification body is expected to interview process owners and sample records to verify that the quality system has been implemented. Certification body auditors will also typically verify that your company has conducted a full quality system audit and at least one management review. Finally, the auditor will usually select a process such as corrective action and preventive action (CAPA) to make sure that you are identifying problems with the quality system and taking appropriate measures to address those problems.
Your goal for the Stage 1 audit should not be perfection. Instead, your focus is to make sure that there are no “major” nonconformities. The term “major” used to have a specific definition:
Absence of a documented procedure or process
Release of nonconforming product
Repeat nonconformities (not possible during a Stage 1)
Under the MDSAP, the grading system for nonconformities now uses a numbering system for grading nonconformities: “Nonconformity Grading System for Regulatory Purposes and Information Exchange Study Group 3 Final Document GHTF/SG3/N19:2012.” Any nonconformity is graded on a scale of one to four, and then two potential escalation rules are applied. If any nonconformities are graded as a four or a 5, then the auditor must assess whether a five-day notice to Regulatory Authorities is required. A five-day notice is required in either of the following situations: 1) one or more findings grading of “5”; or 2) three or more findings graded as “4.” If your Stage 1 audit results in a five-day notice, then you are not ready for your Stage 2 audit. For example, a complete absence of two required procedures in clauses 6.4 through 8.5 of ISO 13485:2016 would result in two findings with a grading of “4.” This would not result in a five-day notice, but the absence of a third required procedure would result in a five-day notice.
The duration of your Stage 1 audit will be one or two days, but a 1.5-day audit is quite common for MDSAP Stage 1 audits. The reason for the 1.5-day Stage 1 audit is that it is challenging to assess readiness for Stage 2 in one day, and if the total duration of Stage 1 and Stage 2 is 5.5 days, then the Stage 2 audit could be completed in four days. The four-day audit is more convenient than a three-day audit for a two-person audit team.
After your Stage 1 audit, you will receive an audit report, and you should expect findings. You should initiate corrective actions for each finding immediately, to make sure the findings are corrected and prevented from repeat occurrence before the Stage 2 audit. The duration between the audits is typically about 4-6 weeks. That does not leave much time for you to initiate a CAPA, perform an investigation of the root cause, and implement corrective action. At a minimum, you must submit a corrective action plan for each finding to your MDSAP auditing organization (AO) within 15 calendar days of receiving the finding. For any findings graded as a “4” or higher, you will need to provide evidence of implementing the corrective action plan to the AO within 30 calendar days of receiving the finding. You are also unlikely to have enough time to conduct an effectiveness check prior to the Stage 2 audit.
Step 6 – Stage 2, Initial ISO 13485 Certification Audit
The Stage 2 initial ISO 13485 certification audit will verify that all regulatory requirements have been met for any market you plan to distribute in. The auditor will complete an MDSAP checklist that includes all of the regulatory requirements for each of the countries that recognize MDSAP: 1) the USA, 2) Canada, 3) Brazil, 4) Austria, and 5) Japan. The auditor will also sample records from every process in your quality system to verify that the procedures and processes are fully implemented. This audit will typically be at least four days in duration unless multiple auditors are working in an audit team.
The audit objectives for the Stage 2 ISO 13485 certification audit specifically include evaluating the effectiveness of your quality system in the following areas:
Applicable regulatory requirements
Product and process-related technologies
All procedures will be reviewed for compliance with ISO 13485:2016 and the applicable regulations. The auditor will also sample records from each process. If the auditor identifies any nonconformities during the audit, it is important to record the findings and begin planning corrective actions immediately. If you have any questions regarding the expectations for the investigation of the root cause, corrections, corrective actions, and effectiveness checks, you should ask the auditor during the audit or the closing meeting. At a minimum, you must submit a corrective action plan for each finding to your MDSAP auditing organization (AO) within 15 calendar days of receiving the finding. For any findings graded as a “4” or higher, you will need to provide evidence of implementing the corrective action plan to the AO within 30 calendar days of receiving the finding. The auditor will not be able to recommend you for ISO 13485 certification until your corrective action plans are accepted.
If you receive a finding with a grading of “5,” or three or more findings graded as “4,” then the MDSAP auditor is required to issue a five-day notification to the regulators. The auditor will also need to return to your facility for a follow-up audit to close as many findings as they can. It is not necessary to eliminate all of the findings in order to be recommended for ISO 13485 certification, but the grading of the findings must be reduced to at least a “3” before recommending the company for certification. The number of findings also determines whether the auditor recommends your company for certification.
In addition to reviewing the findings and conclusions of the audit during the closing meeting, the auditor will also review the plan for the annual surveillance and re-certification with you. Each certification cycle is three years in duration. There will be two surveillance audits of approximately one-third the duration of the combined duration of stage 1 and stage 2 initial certification audits, and the first surveillance audit must be completed within 12 months of the initial certification audit. In the third year, there will be a re-certification audit for two-thirds of the duration of the combined duration of stage 1 and stage 2 initial certification audits. The initial ISO 13485 certificate will be issued with a three-year expiration, and the certificate is typically received about one month after the acceptance of your corrective action plan.
There are no stupid questions, and we can save your weeks of wasted time if you just ask for help. We are always looking for new ideas for blogs, webinars, and videos on our YouTube channel. If you have any general questions about obtaining ISO 13485:2016 certification, please email Rob Packard email@example.com. If you have a suggestion for new ISO 13485 training materials, you can also use our “Suggestion Box.” You can also schedule an initial free consultation with Rob using his calendly link.
What do you look at and look for when you are auditing risk management files to ISO 14971 and the new Regulation (EU) 2017/745?
Your cart is empty
Next week, November 15th @ Noon EST, you will have the opportunity to watch a live webinar teaching you what to look at and what to look for when you are auditing risk management files to Regulation (EU) 2017/745 and ISO 14971. Risk Management Files are one of the essential requirements of technical documentation required for CE Marking of medical devices. Most quality system auditors are trained on how to audit to ISO 13485:2016 (or an earlier version of that standard), but very few quality system auditors have the training necessary to audit risk management files.
Why you are not qualified to audit risk management files
Being a qualified lead auditor is not enough to audit the risk management process. When you are auditing a risk management file, you need risk management training and lead auditor training. To audit the risk management process, you will also need training on applicable guidance documents (i.e., ISO/TR 24971:2020) and applicable regulations (i.e., Regulation 2017/745 and/or Regulation 2017/746). There may also be device-specific guidance documents that specify known risks and risk controls that are considered state-of-the-art.
Creating an audit agenda
Once you have scheduled an audit of risk management files, and assigned a lead auditor, then the lead auditor needs to create an audit agenda. The audit can be a desktop audit that is performed remotely, or it can be an on-site audit. Regardless of the approach, the audit should include interviewing participants in the risk management process documented in the risk management file. As a rule of thumb, I expect a minimum of 30 minutes to be spent interviewing the process owner and one or more other participants. Then I spend an additional 60 minutes of auditing time reviewing documents and records.
Your audit agenda should specify the following items at a minimum:
the method of auditing to be used,
date(s) of the audit,
the duration of the audit,
the location of the audit, and
the auditing criteria.
The auditor(s) and the auditee participants should be identified in the audit agenda. Finally, you should specify which documents and records are required for audit preparation. These documents will be used to help identify audit checklist questions and to determine a sampling plan for the audit. At a minimum, you will need a copy of the risk management procedure and a list of the risk management files that are available to audit. You may also want to request the audit plan for each of those risk management files.
What did you look at and look for during your risk management audit?
When you audit the risk management process, you could take any of the following approaches or a combination of more than one. You could audit the process according to the risk management procedure. You could audit the process according to the risk management plan(s) for each risk management file. You could audit using the process approach to auditing. Finally, you could audit in accordance with specific requirements in the ISO 14971:2019 standard and applicable regulations (i.e., Regulation 2017/745). Regardless of which approach you take, your audit notes and the audit report should identify which documents and records you sampled and what you looked for in each document. Providing only a list of the documents is not enough detail.
Creating an auditing checklist for risk management files
Auditors with limited experience are taught to create an audit checklist by creating a table that includes each of the requirements of the audit criteria. For a risk management file, this would include a list of each of the requirements in ISO 14971 for a risk management file (i.e., Clause 9???). However, this approach is more like the approach that you should be using for a gap analysis. The better approach for creating an audit checklist for risk management files is to start by creating a turtle diagram. In the “process inputs” section (i.e., step 2 of 7), you would add questions derived from your review of the risk management plan(s). In the “process outputs” section (i.e., step 3 of 7), you would add questions specific to the risk management report and other records required in a risk management file. In the “with whom” section (i.e., step 5 of 7), you would add questions related to training and competency. You might also identify additional people involved in the risk management process, other than the process owner, to interview as a follow-up trail. In the “how done” section (i.e., step 6 of 7), you would add questions specific to the procedure and forms used for the risk management process. Finally, in the “metrics” section (i.e., 7 of 7), you would verify that the company is conducting risk management reviews and updating risk management documentation in accordance with the risk management procedure and individual risk management plan(s).
Audits are just samples
Just because you can generate a lot of questions for an audit checklist does not mean that you are required to address every question. Audits are intended to be a “spot check” to verify the effectiveness of a process. You should allocate your auditing resources based on the importance of a process and the results of previous audits. I recommend approximately three days for a full quality system audit, and approximately 90-minutes should be devoted to a process unless it is the design control process (i.e., Clause 7.3 of ISO 13485) which typically requires three to four hours due to the importance and complexity of the design controls process. Therefore, you should schedule approximately 30 minutes to interview people for the risk management process and approximately 60 minutes should be reserved for reviewing documents and records. With this limited amount of time, you will not be able to review every record or interview everyone that was involved in the risk management process. This is why auditors always remind auditees that an audit is just a sampling.
Which records are required in a risk management file?
The contents of a risk management file is specified in ISO 14971:2019, Clause 4.5. There are only four bullets in that section, but the preceding sentence says, “In addition to the requirements of other clauses of this document.” Therefore, your risk management file should address all of the requirements in ISO 14971:2019. What I recommend is a virtual risk management folder for each risk management file. As the auditor, you should also request a copy of the risk management policy and procedure. An example of what this would look like is provided below. The numbers in front of each subfolder correspond to the sub-clause or clause for that requirement in ISO 14971:2019.
Which records are most valuable when auditing risk management files?
As an auditor, I typically focus on three types of targets when auditing any process. First, I will sample any corrective actions implemented in response to previous audit findings. Second, will sample documents and records associated with any changes made to the process. Changes would also include any changes that were made to individual risk management files or the creation of a new risk management file. Finally, my third target for audit sampling is any item that I feel is at risk for safety or performance failures. The severity of the safety or performance failure is also considered when prioritizing audit sampling. In the context of a risk management file, I always verify that production and post-production activities are being conducted as planned. I try to verify that risk analysis documentation was reviewed for the need to update the documentation in response to complaints and adverse events.
More auditor training on risk management files
We are recording a live webinar intended to teach internal auditors and consultants how to perform a thorough audit of risk management files against the requirements of the new European Regulation (EU) 2017/745 and ISO 14971.
Auditing Risk Management Files
In this new webinar, you will learn how to conduct a process audit of risk management files. You will learn what to look at and what to look for in order to verify compliance with Regulation (EU) 2017/745 and ISO 14971:2019. The webinar will be approximately one hour in duration. Attendees will be invited to participate in the live webinar and receive a copy of the native slide deck. Anyone purchasing after the live event will receive a link to download the recording of the live event and the native slide deck.
In addition to this webinar on auditing risk management files, we also have other risk management training webinars available. The webinar on auditing risk management files will be hosted live on November 15, 2022 @ Noon EST (incorrect in the live video announcement).
Process monitoring is required but do you know whether monitoring every procedure is required by the FDA QSR or ISO 13485?
One of the elements that Medical Device Academy has incorporated into each procedure we created in our turnkey quality system is a section titled, “monitoring and measurement.” The purpose of this section is to force each process owner to identify a process metric for monitoring every procedure. In some cases, we suggest a metric that would be appropriate for most companies establishing a new quality system. In other procedures, we use the following default text:
Where are the requirements for process monitoring in 21 CFR 820?
Some of the companies that have purchased our turnkey quality system have asked, “Is it required to monitor and measure something in every procedure?” In general, it is not a specific requirement to have a metric specified in each procedure. In fact, if your quality system is not ISO 13485 certified, there are actually only a few places where the US FDA requires monitoring. The FDA does not have a section specific to monitoring and measurement of processes, but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used:
21 CFR 820.100(a)(1) – “Analyzing processes, work operations, concessions, quality audit reports, quality records, service records, complaints, returned product, and other sources of quality data to identify existing and potential causes of nonconforming product, or other quality problems. Appropriate statistical methodology shall be employed where necessary to detect recurring quality problems;”
21 CFR 820.200(b) – “Each manufacturer shall analyze service reports with appropriate statistical methodology in accordance with § 820.100.”
21 CFR 820.250 – “(a) Where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics. (b) Sampling plans, when used, shall be written and based on a valid statistical rationale. Each manufacturer shall establish and maintain procedures to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed. These activities shall be documented.” Note: the other two instances are the title of 21 CFR 820.250.
The word “monitoring” is equally rare (i.e. 4x) in the QSR:
21 CFR 820.70(a) – “Each manufacturer shall develop, conduct, control, and monitor production processes to ensure that a device conforms to its specifications…Where process controls are needed…(2) Monitoring and control of process parameters and component and device characteristics during production.”
21 CFR 820.75(b) – “Each manufacturer shall establish and maintain procedures for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met…(2) For validated processes, the monitoring and control methods and data, the date performed, and, where appropriate, the individual(s) performing the process or the major equipment used shall be documented.”
Where are the requirements for process monitoring in ISO 13485:2016?
ISO 13485:2016 has a section specific to monitoring and measurement of processes (i.e. Clause 8.2.5). In addition, the word “monitoring” occurs 52 times in the standard and there are 60 incidents of some variant or the exact word. , but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used. There are four Clause headings that actually include the word monitoring:
Clause 7.6, Control of monitoring and measuring equipment
Clause 8.2, Monitoring and measurement
Clause 8.2.5, Monitoring and measurement of processes
Clause 8.2.6, Monitoring and measurement of product
In Clause 1, Scope, and Clause 4.1.5, the Standard states that any outsourced processes remain the responsibility of the company and must be accounted for in the quality system by monitoring, maintaining, and controlling the processes.
Monitoring of risk is included in the definition of “risk management” in the Standard (i.e. Clause 3.18).
Clause 4.1.3 states that the organization shall, “b) ensure the availability of resources and information necessary to support the operation and monitoring of these processes…d) monitor, measure as appropriate, and analyze these processes.”
Clause 4.2.3 states that the contents of the Medical Device File (i.e. MDR or TF), shall include, “d) procedures for measuring and monitoring.”
Monitoring and measurement of processes and product are required inputs to the Management Review in Clauses 5.6.2e) and f).
Clause 6.4.1 requires a procedure for monitoring the work environment if it can have an effect on product quality.
Clause 7.1 requires the company to consider including monitoring in product realization planning.
Clause 7.4.1 requires a plan for monitoring suppliers.
Clause 7.5.1 requires monitoring production and service, including the monitoring of process parameters and product characteristics.
Clause 7.5.6 requires monitoring of validated process parameters.
Clause 7.5.8 requires identification of status with regard to product monitoring and measurement (i.e. inspection status).
Clause 7.6 requires monitoring and measurement of calibrated devices and validation of any computer software used to monitor calibrated devices.
Clause 8.1 states that companies shall plan and implement monitoring and measurement of processes.
Clause 8.2 is titled, “Monitoring and measurement.”
Clause 8.2.1 requires monitoring of customer feedback.
Clause 8.2.5 requires monitoring of processes to ensure planned results are achieved.
Clause 8.2.6 requires monitoring of products to ensure product requirements have been met.
Clause 8.4 requires data analysis of monitoring data from at least six different processes:
Conformity to product requirements
Characteristics and trends of processes and products, including opportunities for improvement
Service reports, as appropriate
In summary, while not every single clause that requires a procedure includes a requirement for monitoring, there are a number of processes where the requirement to monitor the process is explicitly stated.
Why do all of our procedures include the requirement for metrics?
Medical Device Academy expanded the requirement for monitoring to all procedures for five reasons:
Quality objectives must be “established at relevant functions and levels within the organization.” Therefore, establishing monitoring requirements for each procedure ensures that top management has metrics for every process and a lack of data is never an excuse for not establishing a new quality objective when improvement is needed.
If every procedure has a requirement for monitoring, then employees don’t have to remember which processes require monitoring and which processes do not explicitly require monitoring.
The process approach to auditing includes metrics of the process as one of the seven items that are included in every process turtle diagram, and therefore, including metrics for each procedure facilitates the process approach to auditing.
If a company does not have a process metric already established, it is often difficult to perform an investigation of the root cause of quality issues. If a metric is already being monitored for the process, this facilitates the investigation of the root cause and you can use the baseline monitoring data to help you establish effectiveness criteria for the corrective action.
Finally, most companies struggle to identify preventive actions as required by Clause 8.5.3, and we have found that data analysis of monitoring data is the best source of identifying new preventive actions.
What are the disadvantages when you monitor and measure something in every procedure?
The primary reason for resistance to identifying a metric for monitoring in every procedure is that it will increase the workload for the employees responsible for that process. However, monitoring of data does not always increase workload. In fact, when process data is recorded in real-time on a run chart it is often possible to identify a trend much earlier than when data is simply recorded and subjected to monitoring.
Example #1: The automatic tracking of toner in a printer tells HP when to ship you a new toner cartridge before you need it. This ensures that there is no loss in productivity because you never run out of ink or the ability to print documents.
Example #2: Companies will use project management software (e.g. Asana) to monitor labor utilization. This will help identify when a specific resource is nearing capacity. When this occurs, the project manager can add time buffers to prerequisite steps and adjust the starting date of the resource-limited tasks to an earlier starting date. This ensures that more time is available to finish the task or to take advantage of resource availability at an earlier date.
Example #3: Monitoring the revision date for procedures helps the document control process owner identify procedures that should be evaluated for the need to be revised and updated. Often this is articulated as a quality objective of reviewing and updating all procedures within 2 years. This also ensures that procedures remain current and compliant with regulatory requirements.
What are the advantages of monitoring every procedure?
The phrase “what gets measured gets managed” is a popular business philosophy that implies measuring employee activity increases the likelihood that employees will complete a task or perform it well. In contrast, if a process is not monitored, employees may assume that it is not important and the tasks may be skipped or completely forgotten. Setting quantitative goals is also sometimes integrated with economic incentives or bonuses that are granted to individuals and teams.
FDA transition from QSR to ISO 13485
The US FDA is planning its transition from 21 CFR 820 to ISO 13485 as the quality system criteria. This will force companies to make adjustments to their quality systems and increase the amount of process monitoring performed. My general advice is to work with employees that are performing tasks to identify streamlined methods for monitoring those tasks without being overly burdensome. Then you and the employees you manage can analyze the data together and identify opportunities for improvement. When you do this, experiment with manual methods using whiteboards and paper charts that are visible in public areas first. Only implement automated solutions after you have optimized the data being collected and the frequency of data collection, and remember that not every process will benefit from automated statistical process control. Sometimes the simple approach is best.
This article explains why remote audit duration should not exceed 90 minutes and the unique opportunities created by a series of short remote audits.
Parkinson’s Law and the subject of audit duration
On November 19, 1995, Cyril Northcote Parkinson published an essay in the Economist. The title of the article was “Parkinson’s Law.” In the first sentence of the essay, Parkinson says, “It is a commonplace observation that work expands to fill the time available for its completion.” This essay refers to the observation that work is elastic concerning the demands on time when completing paperwork. When I first trained as an auditor, trainers emphasized that the most significant challenge faced by auditors is to complete an audit within the time available. An auditor’s task is to achieve the audit objectives within the time specified by the audit program manager. Time is precious, and you cannot easily extend the audit duration after scheduling the audit.
How much time is needed for a full quality system audit?
This question is a silly question to ask a consultant that works on an hourly basis. A consultant working on an hourly basis will make more money if they work more hours. Therefore, there is little incentive to underestimate the time required to complete the objectives of an audit. However, after completing hundreds of audits, I can honestly state that eight hours is not enough time to perform a full quality system audit of a medical device company’s quality system. However, I completed a full quality system audit of a small company in less than two days. I also had difficulty completing an audit of a larger company in four days. An FDA inspector typically requires four days to complete a routine inspection, even at foreign manufacturers where English is a second language, and they only need to return on the fifth day to prepare their FDA 483 observations to give to the company. Therefore, three days is typically the absolute minimum time required to complete a full quality system audit.
Does Parkinson’s Law apply to audit duration?
Parkinson’s Law certainly applies to the audit duration. If the lead auditor assigns a team member to review the CAPA process, the task is unlikely to be completed in 30 minutes, and most auditors would struggle to appear busy for more than three hours. You need enough notes to provide objective evidence of conformity for your audit report, but if you finish too quickly, then others may perceive that you were not thorough. Therefore, most auditors will begin any process audit by asking for a copy of the procedure and a log of the records available. The auditor will quickly review the procedure’s revision history to determine when the last revision was made and if there have been any significant revisions since the last audit. Next, the auditor will review the log to estimate how many records should be sampled. The auditor will then estimate how much time is needed to review the sampled records. Finally, a quick mental calculation is made to determine how much time remains for procedure review before the auditor must move on to interview the next subject matter expert.
Why are auditors always behind schedule?
An auditor begins with small, close-ended questions that are designed to put the auditee at ease. The auditor may even comment on unrelated subjects to build rapport first. Records may not be readily available, but auditors almost always have to wait for record retrieval. The request is recorded, copies are made, and the subject matter expert may need a little time to review before handing the auditor the requested record. Auditors will ask clarifying questions, and auditees will need a few moments to check their facts. Any one of these delays is insignificant by itself, but collectively there may be two-and-half minutes of delay cumulatively for each record requested if you sample five records, which represents a combined delay 12.5 minutes. If you average only seven minutes to review each record, then a sampling of five records will require 47.5 minutes. This will leave you only 12.5 minutes for introductions, review of the procedure, and conclusions. If you want to interview any of the people that investigated root-cause, then you will need more than an hour to complete your audit, and you will not finish in the one hour scheduled.
Why is it so hard to complete a full quality system audit in three days?
Most of your process audits require a few more minutes than you expected, but you will also need time to walk to the next subject matter expert, or you will be waiting for the next subject matter expert to enter the conference room. If the quality system consists of only the minimum twenty-eight required procedures, your full quality system audit will require more than 28 hours to complete. If there are additional regulatory requirements for CE Marking or ISO 13485 certification, you will need even more time to audit every process. You should also expect certain processes to require more time to properly sample records, such as technical documentation and design controls. Even the most experienced auditors struggle to review a technical file and/or design history file in less than two hours.
What happens to an auditor after auditing all day?
As a Notified Body auditor, I used to leave my home in Vermont on Sunday afternoon and drive two hours to the nearest major airport. Then I would be gone all week conducting audits. On Friday, I would drive home and arrive in the middle of the night. Each day audits would begin early in the morning, and I would complete the day after 8.5 to 9 hours of work. Jet lag, sleep deprivation, too little exercise, and constantly eating at restaurants took its toll. I would consult my Google calendar to learn what city I was in each morning, and to remember what company I was on my schedule for the day. I would purposely try to do as much walking around during the day just to keep my blood flowing and to help stay awake. I would read documents while pacing back-and-forth in conference rooms, and I would always make sure that we had to audit the most remote area of a facility after lunch to make sure that I didn’t fall asleep. I will tell stories and jokes to entertain my hosts, but it was necessary to break up the monotony of auditing quality systems seven days a week. I would make sure I drank at least six liters of water each day for health, but this also gave me an excuse to go to take frequent bathroom breaks. Somehow I managed to survive that lifestyle for more than three years. Each day my feet, legs, back, and neck were in severe pain. I had constant headaches, and I know the quality of my work gradually declined throughout each day. The most valuable lesson I learned was, you need to move frequently, or you will die.
What happens when you sit in front of a computer for eight hours?
I can sit in front of a computer longer than almost anyone I know. When I focus on work, four hours can elapse without me getting up from a chair even once. I might pick up my empty coffee mug four or five times to take a sip before I am conscious of the need to get another cup. On days where my schedule consists primarily of Zoom meetings, I may sit through as many as six consecutive meetings before I take the time to get up and go to the bathroom and get a drink of water. Clients may perceive that I have tremendous endurance, but there are negative consequences to this work pattern. My wrist becomes sore, and I need to switch my mouse pad and the style of the mouse I am using every day. I change computers, switch microphones, and take a short walk. My neck, back, and legs will hurt worse than any of the audits during my years as a Notified Body auditor. Sitting at a computer all day has resulted in mild symptoms of restless legs syndrome. Sitting at a computer continuously for the audit duration is physically exhausting and tedious. If you must complete a remote audit on a continuous eight-hour day, you can, but it is not healthy or productive. The negative health consequences and negative impact on productivity are equally applicable to auditees.
What can you do to reduce audit fatigue during a remote audit?
The most straightforward strategy for reducing fatigue is to take breaks. Instead of auditing for eight hours continuously, try auditing in two or three 90-minutes segments each day. If you are auditing someone in a different time zone, you may only be able to accommodate an audit duration of one 90-minute session per day without working through the night. Taking breaks will allow you to leave your computer, eat food, and even go to the bathroom. You can recharge your headset during a break too. You should consider taking a walk outside. It is incredible how much better you feel when you get some exercise, stretch, and experience a little natural light instead of the unnatural glow of your computer’s monitor. The person you are auditing will appreciate the breaks, but they will also enjoy the improvement in your overall demeanor. A simple smile after a 30-minute break has a tremendous positive impact.
How can we utilize breaks more effectively during remote audits?
Auditors need documents and records to review as objective evidence. The most obvious way to make use of breaks is for the auditor to give the auditee a list of documents and records to gather during the break. This will give the auditee an excuse to go and get the documents and records if they are stored in another location. The auditee might also scan records during a break. A break also gives subject matter experts time to re-familiarize themselves with the documents and records before resuming the audit. Auditees and auditors will need to recharge batteries, but the auditor might take time to convert their notes into a summary for the final audit report. The auditor might also review the audit criteria one more time before writing a nonconformity. The auditee might take advantage of the break to initiate a new CAPA and write a draft of the corrective action plan. Then when the audit resumes, the auditee can review the draft plan with the auditor to ensure that the plan is appropriate and nothing was accidentally omitted from the CAPA plan.
Why are 90 minutes a magical audit duration?
Auditing one process in a single 45-60 minute session is ok, but if you audit two processes in a single 90-minute session, you can reduce the time spend starting and stopping the audit session by half. Adding a third process to a single session will have a smaller impact, and the meeting will need to be so long that most participants will begin to lose concentration, and fatigue becomes a significant factor. Ninety minutes is not quite long enough to audit two processes effectively. Still, an auditor can request procedures in advance of the session or spend time after the session reviewing procedures. Therefore, by paying an additional 30 minutes reviewing two procedures “off-line,” the auditor can dedicate 100% of the “on-line” time to reviewing records and interviewing subject matter experts. The result is a fast-paced, 90-minute session where each subject matter expert typically is only needed for 45 minutes. Alternatively, if you are auditing more complex records like a design history file, you can spend all 90 minutes discussing that area.
If you are planning a remote audit, you will need more remote audit resources than a webcam and web conferencing software. Matthew Walker is a significant contributor as co-author of this article.
Clause 5 of ISO 19011:2018 is titled “Managing an audit program,” and subclause 5.4.4 is specific to determining audit program resources. For conducting audits remotely, you will need remote audit resources. Almost every laptop has a built-in webcam and microphone, and that is the minimum functionality you will need to conduct a remote audit. However, adding other software and hardware technology can improve the efficiency and effectiveness of your audit team.
What remote audit resources do you need?
Remote audits are not the same as a desktop audit, because a remote audit requires remote access to more than emails containing procedures and records. Auditors need access to people and access to physical areas of your facility. This creates one of the most significant challenges for this type of audit method. Call me a Negative Nancy, but I suspect that most audit plans do not specifically include logistical preparations to support this audit method. On the surface, it seems like a simple concept. Internet access and a scanner should cover most of the needs for the auditee to survive this digital encounter. In practice, conducting a remote audit that genuinely adds value and does more than checkboxes, requires serious planning.
Let’s start with the obvious; a remote audit needs a way for the auditor and the auditee to communicate with each other. Ideally, you need more than your phone. We recommend Zoom for video conferencing, but we list several other video conferencing software applications below. Here are the features of Zoom that we typically use during a remote audit:
Video Chat –Using Zoom, two or more parties can communicate using video input from webcams. This is nice because it allows for a more visual conversation, and you can see more of the facial expressions and body language of the person you are speaking with than you can with a traditional phone call. It also allows for sign language to be used if necessary.
Screen Sharing – Screen sharing is an essential tool you will use during a remote audit because it allows you to share documents and records on your screen even if you are not the host. The more records you have electronically, the more valuable screen sharing will be during the audit. An auditor can say, “Can you show me that quality system certificate again?” or “Can you show me where Isomedix is on your approved supplier’s list?” Being able to facilitate those verification activities saves the auditee the hassle of emailing documents or uploading content to a shared folder. This ability to share your screen is also essential for an auditee to demonstrate training effectiveness and competency.
Recording – Meetings can be recorded in their entirety or sections. This allows the auditee to record the opening or closing meetings of the audit to share with others that were unable to attend. If there are questions regarding non-conformities or opportunities for improvement, a recording of the conversation ensures that the auditor has an accurate record of complex objective evidence that would slow down the audit and gives managers a perfect record to demonstrate the issue when corrective actions are initiated.
Chat Record – Zoom, and most other video conferencing software, provides a chat box that can be used to take notes. If someone runs to the bathroom, and you don’t want to forget your question, you can enter it in the chatbox. Chat boxes are especially helpful when there is a language barrier, or someone’s accent is hard to understand. Text typed in the chatbox also serves as a place to record information that may be difficult to remember if you cannot access your audit report. If a production area has too much background noise, the chat feature might be the best way to communicate important details, such as: “That information is found in section 7.5.6 of the Quality Manual; POL-001 rev A.” The chatbox can also be used to communicate a list of documents, or records in a specific date range, that you want an auditee to make available for you to review off-line. Other participants observing the audit may also be responsible for collecting those documents in real-time to ensure the audit can continue without any delay. Finally, content in the chatbox can be recorded as a text file automatically.
Tour Guide –Video chat allows auditees to bring auditors into physical places of their facility as if the auditor were there in person. Production employees can be interviewed, in person and in real-time, while the employee demonstrates processes. You can show how nonconforming materials are labeled and segregated to keep them from accidentally being used for production. When requesting this audit method in an audit agenda, the lead auditor should recommend a dedicated “camera person” with a mobile phone and selfie stick, because it is challenging to answer auditor questions and operate a video camera simultaneously. Remember, remote audit resources consist of hardware, software, and people.
My favorite remote auditing tools (hardware)
My favorite hardware resource is the Pixelbook that I am using to write this article. We write audit reports with Google Docs instead of Microsoft Word because multiple team members can simultaneously edit the same document without creating conflicting versions. We operate Zoom video web conferencing software to speak with auditees and clients, but we use the Pixelbook to type our notes and audit reports. The Pixelbook is lightning fast, and it is a little smaller, so there is just enough room on your desk next to a laptop. The most significant advantage of using Google Docs is realized when you are the lead auditor of an audit team. As a lead auditor, you can type notes in the section of the audit report that other team members are working on, to make sure that they include audit trails from other members of the audit team. This is also an extremely useful technique when you are training a new auditor, and you want to guide them without disrupting the flow of an interview with a subject matter expert.
My second favorite hardware resource is an HD webcam mounted on a flexible arm with a clamp (see picture above). The video quality is 1080p instead of the 720p that is typical of a laptop camera. The flexible arm is equally essential because you can look directly at the camera while I’m simultaneously looking at the monitor. The only thing I dislike about the webcam I am using is the audio quality. Therefore, I use a gaming headset with a microphone to record the audio, so I can hear the people I am interviewing better. Another alternative is high-quality microphone and headphones, as typically seen in use by podcasters. Even though the sound quality is ideal with a separate microphone and headphones, the cost is higher than most gaming headsets, and you will be tethered to microphone–either physically or at least virtually by the need to maintain a consistent distance between your mouth and the microphone. The more hours you spend at the computer, the more you will appreciate the ability to stand up, adjust the camera, and move your legs a little.
Finally, the last piece of essential remote auditing hardware is your mobile phone. Even with a desktop running Zoom, and a Pixelbook running Google Docs, I still need to ask audit team members questions and conduct quick internet searches. Therefore, your mobile phone is essential to keep with you, in silent mode, during your audit. If you don’t have your phone, then you need to stop sharing your screen and send a message during your audit. Your phone is much less disruptive. I use the phone to keep track of time, to set reminder alarms, and to send Slack messages with other people. You can also join a separate Zoom session on your phone, where an audit team member may need you (the lead auditor) to provide input on objective evidence or evaluation of conformity regarding specific quality system requirements. You might also want to take a quick picture of something you observe on video during the audit. If you record the Zoom session, you can always extract a still image, but taking a picture with your mobile phone is more convenient and takes less time. You can then share the image with a Google Drive folder for your remote audit and copy the image into your audit report. As they say, a picture is worth 1,000 words.
One last note on hardware: a 48” flat screen is great for virtual bike rides on your trainer (as seen in the picture above), but it’s just a little too big for a desktop monitor. It’s excellent for side-by-side viewing, but dual monitors are a better approach.
Remote Auditing Resources for Web Conferencing
Currently, we are using Zoom as our video web conferencing software. Still, we used to use GoToMeeting, and there is very little difference in the functionality of the two software platforms. One of the consequences of the COVID19 pandemic is that everyone is more familiar with web conferencing software. Here are a few other options you could consider, including Slack, which we use as a messaging tool, and we have integrated with Zoom within our team’s channel.
Currently, we are using Calendly as the automated appointment scheduling software application for our consulting business. However, the functionality of software applications has changed dramatically in the past few years with better integration tools, such as Zappier. Therefore, don’t be surprised if we change to one of the applications listed below. These applications allow you to manage people, equipment, and conference rooms, but you can also integrate these applications with accounting business processes.
We hosted three international training workshops, and we record training videos for medical device companies every week. Therefore, we gradually accumulated all of the accessories listed below. Technology gadgets for recording videos are continually changing, and our best advice is to save your money. Instead, rely upon a mobile phone and an extra person with “the original selfie sticks” (i.e., arms). Once you complete your first remote audit, then you can think about which of the latest gadgets might make your life easier.
If you have any suggestions for additional hardware and software for remote auditing, please add a comment to this article so we can keep this up to date with the latest technology.
Future Articles & Webinars
Thank you for reading. This article is our third in a ten-part blog series specific to remote auditing techniques: