Quality Management System Archives

May 3 2023

Incoming Inspection – How to perform a single process audit

1 Comment / Auditing, Quality Management System / By Robert Packard

The incoming inspection process is my favorite process to audit, and it is the best process for teaching new auditors.

The above video demonstrates how to use a turtle diagram to conduct a process audit of the receiving inspection process. However, this article goes into more detail. You will learn what to look at and what to look for in each part of the audit process.

Preparation for your audit of incoming inspection

If you are conducting an audit of an incoming inspection, you will need a copy of the procedure (i.e., Receiving Inspection Procedure, SYS-033).

Do you need an opening meeting?

Opening meetings are not required for first-party (i.e., internal) and second-party (i.e., supplier) audits. Only third-party auditors are required to have a formal opening meeting. Having an opening meeting is always a good idea, but keep it brief and use a checklist. Try to set the tone for the audit with your opening meeting. This will be your second impression because you already had a conversation with the process owner in preparation for the meeting. However, you want to give everyone present for the opening meeting that you exhibit all the personality characteristics of a good auditor as defined by ISO 19011:2018. Professionalism, organization, and integrity should be obvious to everyone in the room. However, don’t forget to smile and be polite because your auditee might be very nervous. FDA inspectors seem to have an unwritten rule book (i.e., in addition to QSIT) that encourages them to intimidate the companies they inspect.

Step 1 – “Briefly, please describe the incoming inspection process.”

The purpose of this section is not to duplicate the level of detail found in the procedure. It is meant to provide a brief description of the process. Ideally, you want to write a single sentence for the incoming inspection process’s what, where, when, who, and how. A maximum of five sentences is needed to answer those five questions. The process owner should provide the description, and there is no need for them to go into extreme detail because you have at least six more questions to ask (see steps 2-7 below). If you are doing a supplier audit or an audit of a company you don’t work for, you might want to have a few “ice breaker” questions that precede this question. For example, you might ask the person’s name, title, and the number of years they have worked for the company. You might also consider stealing my favorite auditor disclaimer, “If you see me writing furiously, don’t worry. I’m required to write down objective evidence supporting conformity with requirements. If I start asking the same question three different ways, and I’m not writing any notes, that means I am having trouble finding evidence of conformity, and I need your help.”

Step 2 – “What are the inputs that trigger incoming inspection?”

Inputs and outputs of any process refer to both information and physical items. For 100% administrative processes, you may not have any physical items. Incoming inspection, however, has physical goods you receive from suppliers and inspecting. Therefore, the process inputs you are looking for are physical goods and quality system records associated with those goods. For example, if a bunch of titanium round bars were ordered by a buyer in your purchasing department, the physical goods are the titanium bars. The purchase order is one of the quality system records. Other input records that are usually requested to be shipped with the titanium include a packing slip, a certification of analysis, and a dimensional inspection report. It is common to see the incoming inspection activity be delayed because the records are not included with the shipment from the supplier. One recommendation for a process improvement is to require the supplier to send records electronically at the time of shipment instead of sending hardcopies with the product. Statistical inspection sampling plans and work instructions are often confused with input records. These documents are needed to start the incoming inspection, but these are documents that belong in step six of the turtle diagram.

Step 3 – “What are the outputs of the incoming inspection process?”

After incoming inspection is completed there is a requirement to identify the status of the physical product (i.e., accepted or rejected). Usually, a green tag will be used to identify the product as accepted. The tag will also identify the part number, lot, and quantity of product accepted. If the product is titanium, each bar will get a tag. The product will then be transferred to a designated storage area. If you are conducting an audit of a supplier, or a full quality system audit, auditing the warehouse for storage and handling processes is a logical next process. The auditor should look for whether product is segregated in designated locations for specific types of product or if the storage locations are “random” but identified electronically in a material resource planning (MRP) system. The quality system records output from the incoming inspection process will be inspection records and either a green release tag or red rejection tag. If the product is rejected, the product shall be transferred to a quarantine area for nonconforming product and a nonconforming material record (i.e., NCMR) is initiated. Therefore, the process for controlling nonconforming material is another process that could be a logical next process to audit.

Step 4 – “What resources are needed for this process?”

This part of the process approach to auditing is one of the most neglected parts of the quality system. Resources include the facility infrastructure, manufacturing equipment, measurement devices used for inspection, and quality system software used to maintain records of incoming inspection. In this part of the process audit the auditor must be observant. Maintenance records might be located on the side of equipment and they can be reviewed as the auditor walks through the area. This would be an opportunity to interview personnel to make sure they can explain the maintenance process and the equipment maintenance is being performed as planned. The auditor should also determine if equipment validation is required. If the equipment is automated (e.g., automated optical inspection), then an installation qualification (i.e., IQ) should be requested as a quality system record to review at the end of the process or as part of the process for process validation. If the inspection area includes a metrology lab, then the environment may be temperature and humidity controlled. In these types of environments, records of environmental monitoring and trending of environmental conditions should be verified. Lighting, magnification, and particulate filtration could be other environmental requirements for the inspection area. Pest control should be verified in the receiving area, inspection area, and storage areas. The receiving area and warehouse storage are common areas to find pests. Calibration identification should be recorded as a potential follow-up trail for any measurement devices used in the inspection area, and if software is used you will want to verify that quality system software tool validation has been performed.

Step 5 – “Who performs this process?”

A combination of three different roles and responsibilities are typical for this process: 1) department manager, 2) receiving personnel, and 3) inspection personnel. Sometimes one or more of these roles will be combined into one job. The activities sometimes are only performed for a few hours each day, and the personnel that perform the incoming inspection process are assigned to other roles, such as warehouse storage, handling, and shipping. Auditors should always try to interview one or more of the people doing the receiving and inspection activities instead of limiting the interviews to the process owner. Often I will ask the personnel to demonstrate the receiving process and the inspection process. In order to make sure this is possible, you will need to communicate that you want to observe these activities prior to the audit or during the opening meeting. If you don’t, the receiving and inspection activities may already be completed before you start to interview the personnel. Any personnel that are unable to explain the tasks they perform may be targets for verification of training records, effectiveness of training, and competency.

Step 6 – “How is this process performed?”

If an auditor interviews personnel, most people will describe the process in a very haphazard way and steps will be missed. This is why asking people to demonstrate the process is better. The best method is for the person to access the current, approved work instruction or procedure for the process. Then the person should follow the work instruction step-by-step. This allows the person to use the work instruction or procedure as a “crutch” and reduces their nervousness. This also eliminates the skipping steps if the procedures and work instructions are sufficiently detailed. Any blank forms used and statistical inspection standards are also considered quality system documents that define how the process is performed. Sometimes the process owner will provide these documents during their interview, and other times this documents are provided as audit preparation documents. If the documents are not provided in advance the auditor should make sure that they review the documents during observation of activities being performed. This is where an auditor may identify the use of obsolete quality documents, missing details in the documents, and details that are inconsistently followed by personnel.

Step 7 – “What metrics are important for this process?”

Whenever I ask, “What metrics are important in this process?” I typically get a blank stare. Hundreds of business management leaders subscribe to the concept of “what gets managed gets done.” You are also required to establish metrics for your quality system processes in accordance with Clause 8.2.5. Therefore, you need to establish at least one metric, if not more than one. Auditing can help identify opportunities for improvement (OFI), but metrics are the best source of OFIs for a quality system.

Do you need a closing meeting?

You should always conduct a closing meeting for your audits. However, it is also a best practice to summarize your findings for the process owner before you move on to the next process. If some records remain to be reviewed, ensure the process owner knows that the audit results are pending an outcome of reviewing the remaining records. Consider adopting the “sandwich” approach to presenting your findings: 1) something positive, 2) any nonconformities, and 3) something positive. The approach sandwiches the “bad news” between two pieces of “good news.” If you are working as part of a team, the lead auditor should always be aware of the results of your audit. The manager responsible for the process (i.e., the process owner) should also be aware of the results. Do everything you can to prevent unpleasant surprises at the end of the audit.

When you describe any nonconformities, make sure that you include all of the following information:

the grading of the finding (i.e., MDSAP scoring or Major/Minor)
a single sentence stating the finding
the requirement, including a reference to the applicable regulation or standard
objective evidence from your notes

Whenever possible, email a draft of the wording for your nonconformities to the process owner so they can be prepared with clarification questions during the closing meeting. Make sure you agree with your lead auditor before sending the wording of the finding, and copy them on the email communication. If the process owner has initiated immediate corrective action(s), make sure you note this in your report.

Finalizing your audit report

If you are conducting a supplier audit, you need to give the supplier formal feedback from the audit. You will need an audit report for your quality system records, but you are not required to give the supplier the full report. You might provide a summary of the audit for the supplier instead. If you do this, you should include a copy of that communication in your quality system record (e.g., an appendix to your audit report). If you are going to provide a summary of findings, the content should include at least the following:

positive findings (i.e., strengths)
negative findings (i.e., weaknesses)
nonconformities (if any)
required actions (e.g., supplier corrective action plan)
due date(s) for objective evidence of containment, corrections, and corrective actions
recommendations for follow-up (e.g., next audit)

If you prepare an internal audit report, all of the above content should be included. However, the report should have additional details:

audit purpose
audit scope
audit date(s)
audit criteria
name of participants
date of report
closure of previous audit non-conformities
reference to the audit agenda
deviations, if any, from the agenda
summary of the audit, including any obstructions
objective evidence sampled (i.e., what you looked at and what you looked for)
opportunities for improvement (if any)

Incoming Inspection – How to perform a single process audit Read More »

Apr 4 2023

OpenAI and Elsmar never trust their help with regulatory questions?

5 Comments / FDA, Quality Management System, Registration, Regulatory Consultant / By Robert Packard / Elsmar Cove, QA information resource

Everyone has a favorite resource they use to answer regulatory questions, but can you trust OpenAI or Elsmar to answer correctly?

If you are deathly afraid of trying new technology, the image above is a screen capture from OpenAI describing “itself.” OpenAI is artificial intelligence (AI), but it is not self-aware yet. The image below is a screen capture from the “About” webpage for Elsmar Cove. This article was the oldest post on the Medical Device Academy, and it described how to use Elsmar Cove as a resource for quality systems and regulatory questions. To update that blog, we are comparing the use of OpenAI with Elsmar Cove. Just in case you were wondering, Elsmar Cove is #6 on our list of favorite search tools, and OpenAI is #5:

Ask the Medical Device Academy team – 1st place, you can email Lindsey Walker or call 802.989.3939
Use Basil Systems software we license – 2nd place, you can email Mike Belongie or call 612.237.3404
Search the FDA.gov device databases – 3rd place, you can email DICE or call 301.796.7100
Search Google (world’s #1 search engine) – 4th place, ask any 5-year-old for help

Are the answers provided by OpenAI and Elsmar Cove accurate?

To test the accuracy of a common regulatory question, we chose a question we weren’t 100% sure about when a client asked last month. I asked my team, but nobody was 100% certain. Basil Systems is limited to submission and post-market surveillance data. I searched FDA.gov, but it was not clear. Google gave us a link to the FDA website. I asked a couple of ex-FDA consultants, but they gave me outdated information. On Thursday, March 30, 2023, I asked Lisa King during an AAMI course I was co-teaching. Lisa is a Consumer Safety Officer at the FDA responsible for reviewing device entries into the FDA. She is also in very high demand for public training courses. She said the contract manufacturers used to be exempt from registration if they shipped to a legal manufacturer first. The regulations changed, and now 100% of contract manufacturers making a finished device must register with the FDA. She also clarified that the FDA doesn’t use the term “legal manufacturer.”

As you can see from the above answer provided by OpenAI, the ChatGPT engine [i.e., Model: Default (GPT-3.5)] effectively produces the correct answer. Using the same wording for the regulatory question, “Does a foreign contract manufacturer need to register with the FDA if they are shipping the medical device to the legal manufacturer first before the device is exported to the USA?” there were no results from Elsmar Cove. After several attempts, I found what I was looking for using the following search terms, “FDA registration of contract manufacturers.” There were multiple related search results, but the most useful discussion threads in the Elsmar Cove discussion forum were:

#4 “Medical Device Contract Manufacturer – Does the CM need to register with FDA” (March 6, 2021)
#5 “Medical Device Contract Manufacturer – Does the CM need to register with FDA” (February 23, 2021)
#19 “Requirements for Contract Manufacturing of a Medical Device” (November 10, 2017)

The most succinct correct answer in the forum is copied below.

Can you trust OpenAI and Elsmar Cove to answer your regulatory questions?

OpenAI is only as effective as the data used to train it. This is constantly evolving, but we have identified search results that were 100% accurate, results that were outdated, and results that were scary wrong. The same is true of discussion forums. Elsmar Cove is one of the best discussion forums for the medical device industry, but people also use ASQ, RAPS, and AAMI. The quality of the information provided depends upon the knowledge and experience of the people participating in the forum, but it also depends upon the forum’s moderation. Elsmar Cove has some experienced moderators with decades of experience. There is always the chance that the most experienced person in the world could answer your regulatory question incorrectly. This usually creates a problem because everyone else in the forum hesitates to challenge a recognized expert. Therefore, regardless of which resource(s) you use, always try to get a reference to the trustworthy source of the applicable regulation. Even Lisa King could make a mistake, but she immediately said, you can find the regulations in the US Code of Federal Regulations (i.e., 21 CFR 807). The bottom line is, always do your fact-checking and reference your source(s).

OpenAI and Elsmar never trust their help with regulatory questions? Read More »

Mar 28 2023

Nine easy ways to organize and improve quality system procedures

2 Comments / ISO 13485:2016, ISO Certification, Quality Management System / By Robert Packard / PDCA, Standard, Writing Procedures

Would you like to learn nine ways to improve your quality system procedures? One method is precisely the opposite of our advice from 2011.

During a CAPA course I taught on Friday, January 28, 2011, one of the attendees asked if we teach a course on “How to write better quality system procedures.” Unfortunately, we could only offer material from a course about “Training the trainer.” That “Training the trainer” course focused on visual communication. Several books related to Lean Manufacturing explain how to use visual communication to replace text (i.e., “a picture says a thousand words”). During my ride home, however, I thought of a few other ideas that might help anyone writing or re-writing a procedure. The article was updated and posted as a new blog on Tuesday, March 28, 2023.

1. Use a standardized template for your procedures

In 2013 we published a blog about using a procedure template where we described our 12-part procedure template (i.e., TMP-001). You don’t have to mimic our template, but using a template will accelerate the speed of your writing when you create procedures, and it makes sure you don’t forget any of the essential elements. In addition, using templates ensures a consistent format that makes it easier for everyone to find the information they are looking for. Just make sure that your document control procedure allows flexibility to deviate from the template. The ISO 13485:2016 standard does require a “mandatory” format. Referring to your template as “suggested formatting” will avoid unnecessary nonconformities.

2. Create a process “turtle diagram” for each quality procedure

All of the procedures that Medical Device Academy created have a flow chart at the beginning of the procedure showing the procedures and forms associated with processes that are inputs to that procedure and outputs from that procedure. To systematically improve our procedures, we will be systematically replacing those flow charts with turtle diagrams for each process. This will give more detail than our current flow charts, and internal and external auditors can use the turtle diagrams to understand process interactions.

3. Avoid making unnecessary references to regulations and standards

If you are writing a procedure on risk management—it makes sense to reference ISO 14971. It does not make sense to reference all the other risk analysis standards unless you specifically use them to perform risk analysis. ISO 14971:2019, Clause 4.1, also states that you “shall establish, implement, document, and maintain an ongoing process for” risk management activities. However, the ISO 14971 standard is not directly linked to other procedures. Therefore, ISO 14971 should only be referenced in another if you are using it in that procedure or referencing it directly. For example, the Quality Manual (i.e., POL-001) explicitly references ISO 14971. In contrast, the design control procedure (i.e., SYS-008) references the risk management procedure (SYS-010) but doesn’t reference ISO 14971.
Concerning regulations, you should only reference regulations if the procedure meets a specific requirement. Color coding with symbols should demonstrate traceability to requirements (see method #5 below for further explanation). Rather than adding a reference to regulations in a procedure where there is no requirement, a better approach is to indicate in the Quality Manual that only procedures that have specific requirements will reference the regulations, such as 21 CFR 820 or Part 1 of the Canadian MDR.

4. Track standards, regulations, and the version used in your procedures

In the original 2011 version of this article, we advised quality managers to “avoid including the revision of a standard” because “this is just another opportunity for unnecessary nonconformities.” However, we find that our team has trouble identifying every procedure that a change in regulation or a standard might impact. A systematic process is needed to identify every procedure referencing a regulation or standard. Therefore, we will reference all impacted procedures next to the regulation or standard in our Master Document List (i.e., LST-001). References to the regulations will be added to the main tab for policies, procedures, and work instructions (i.e., [POL, SYS, and WI]). References to the standards will be added to the tab for documents of external origin (i.e., [Doc Ext Origin]).

Many people feel that you should not reference the version of a standard in a procedure because adding the version of the standard increases the number of documents that need to be updated when a standard changes. However, if you are only referencing standards in procedures when it is necessary, then that procedure should be reviewed and updated for the need to be changed. Updating the version of the Standard referenced is the best way to document that a gap analysis against the new version has been completed and the necessary updates were made to the procedure.

5. Use color coding and symbols in your quality system procedures

Matthew Walker, Medical Device Academy’s manager of the human factors team, has systematically updated many of our procedures to the EU Medical Device Regulations 2017/745 and the In Vitro Diagnostic Regulations 2017/746. When he updates our procedures, he references the regulations and applicable ISO 13485:2016 clauses. During certification audits, certification body auditors sometimes have difficulty finding where specific requirements are located in the procedures. Therefore, Matthew added color-coded clause references for our clients and auditors as a corrective action. To make the procedures inclusive for people that are color-blind, Matthew added symbols to supplement the color coding. The extra addition of symbols has proven invaluable because now anyone can search the documents electronically for a symbol to find where all the references are located.

6. Indicate the process owner and training requirements associated with each procedure

Identifying the process owner and training requirements in every procedure makes it easier to define who is responsible for reviewing and revising procedures. For the training requirements, the process owner should specify who needs to be trained on the process. Why? They know the procedure best. If there is a “grey area,” this should be resolved with the department manager for the job function. In addition, retraining requirements should be specified. The training section should also clarify if retraining is required when revising a procedure. If the revision is minor, training should only be necessary for people not trained on a previous revision.

7. Adopt the Plan-Do-Check-Act (PDCA) model for the structure of quality system procedures

For the “Plan” portion, the procedure should explain how to prepare to do something. This planning activity can apply to anything from planning to perform an audit to planning to inspect incoming raw materials. The “Do” portion is what most people refer to as the “Procedure” section. The “Check” portion of the procedure is a great place to specify the monitoring and measurement requirements for the process (see Section 8.1 of the Standard). Finally, the “Act” portion of the procedure should indicate what to do when target metrics are unmet. For example, what should be done when an alert limit is reached? What should be done when an action limit is reached?

8. Include the revision history of quality system procedures

It’s helpful to know which Document Change Notice (DCN) approved the document revision, why the changes were made, the nature of the changes, whether there is a related corrective action, and when the change was made. This will also tell auditors whether there is anything new to audit since the previous internal or external audit. This section is usually near the beginning of our procedures, but it doesn’t matter if the revision history is at the end or the beginning. However, it does help to be consistent.

9. Identify the form number, location, and retention period for each record

We have a section about quality system records near the end of every procedure. This section lists each quality system record that is associated with the procedure. The relevant form is referenced, but we recommend storing these records in electronic or paper folders labeled with the form number. If the files are digital, a hyperlink should be included. If the files are paper, then you should list the physical location of storage. The retention period can be listed in each procedure. Still, it will be essential to ensure that this information matches the regulatory requirements and record retention requirements in your “Control of Records” procedure (i.e., SYS-002).

Nine easy ways to organize and improve quality system procedures Read More »

Jan 31 2023

ISO 19011 – Do you need this quality system auditing standard?

5 Comments / Auditing, ISO Auditing, Remote Auditing / By Robert Packard / Audit, ISO 19011, PDCA

Read this article to learn why ISO 19011 standard is a vital guidance for anyone that audits quality systems or manages an audit program.

What is ISO 19011?

ISO 19011 is a seven-part international standard for auditing management systems. The standard defines the eight principles of auditing (e.g., the process approach to auditing), provides guidance on managing audit programs and conducting audits, and includes recommendations for evaluating people for competency. There is also an appendix with details on conducting on-site and remote audits.

If you have ever taken a lead auditor course for ISO 13485, or one of the other quality management system standards, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Quality Management Systems.” In 2018, ISO 19011 was updated, and the changes were not superficial. If you need to purchase a copy of ISO 19011:2018, the Estonian Center for Standardization and Accreditation is the least expensive source we know.

ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting internal and external audits, and determining auditor competency. One of the most common points of confusion in the lead auditor course is the difference between first, second, and third-party audits. In the first edition of this Standard, the difference between first, second, and third-party audits was just a note at the bottom of page one and the top of page two. The note was also not clear. In the second edition of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear. Table 1 was modified further in the 3rd edition to include a bottom row that remains unchanged in the 3rd edition, released in 2018.

Figure 1, found in Clause 5.1 of the 2nd edition, was combined with Figure 2, found in Clause 6.1 of the 2nd edition. The combined figure is now Figure 1 in the 3rd edition. The combined scope of Figure 1 is now a “Process flow for the management of an audit program” and a “Process flow for conducting an audit.” The figure categorizes the various stages of audit program management and conducting an audit into the Plan-Do-Check-Act (PDCA) cycle. We highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard.

The 2018 version still includes an opening meeting checklist (i.e., Clause 6.4.3) and a closing meeting checklist (i.e., Clause 6.4.10). Figure 3 in the 2nd edition, “Overview of the process of collecting and verifying information,” was a poor example of a flow chart. The committee did not update the figure when the standard was updated for the 3rd edition. Therefore, we updated the figure below to provide additional traceability to the Clauses of the Standard. If you incorporate this figure into your quality auditing procedure, you should substitute references to your procedure’s sections instead of the clauses of the standard.

Competency Requirements in ISO 19011

Many audit procedures neglect to define the qualifications and methods for determining the competency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures we read include qualifications for a “Lead Auditor,” but we seldom see anything regarding competency. Unfortunately, this Standard only explicitly addresses the “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When we teach people how to be Lead Auditors, we spend more than an hour on this topic alone.

The Standard would be more effective by providing an example of how third-party auditors become qualified as a Lead Auditor. Third-party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meetings, conducting the audit, closing meetings, and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e., – Stage 2 certification or re-certification), and another qualified lead auditor must evaluate you and provide feedback.

Appendices in ISO 19011

The appendices were the last significant additions to this Standard in 2011 (i.e., 2nd edition). Annex A provided examples of discipline-specific knowledge and skills of auditors. This section was eliminated from the 3rd edition of ISO 19011:

“Due to the large number of individual management system standards, it would not be practical to include competence requirements for all disciplines.” – Copied from the Foreward

I think providing adding a short Annex to each management system standard that defines recommended discipline-specific knowledge would be helpful. Still, that kind of change would need to be initiated with the next version of ISO 9001.

Appendix B in the 2nd edition is now Appendix A in the 3rd edition of ISO 19011. A table (Table A.1 – Audit Methods) compares conducting on-site and remote audits. We were pleased to see that conducting interviews is a significant part of remote auditing in this table. Section A.17 in the appendix provides suggestions for conducting interviews. Still, if you exhibit all 13 professional behavior traits found in Clause 7.2.2, you don’t need advice on speaking with people. For the rest of us mortals, we could use a five-day course on interviewing alone. To improve your skills in this area, ask an experienced auditor with solid interviewing skills to watch and comment on a recording of a virtual audit you perform. Watching yourself audit is cringe-worthy, but we guarantee you will improve.

What are the primary changes to the 2018 version of the standard?

There are seven main differences between the second edition, published in 2011, and the third edition of ISO 19011, released in 2018:

addition of a seventh principle of auditing in sub-clause 4(g) (i.e., risk-based approach);
more guidance on audit program management in Clause 5, including audit program risk;
expansion of Clause 6 on conducting an audit–especially Clause 6.3 on audit planning;
expansion of auditor competence requirements in Clause 7;
updating of terminology to emphasize processes rather than objects;
removal of an annex containing competence requirements for specific quality management systems;
expansion of Annex A to include guidance on new auditing concepts such as remote audits.

Risk-based auditing is the most significant change in the 2018 version of ISO 19011

One of the main differences between ISO 19011:2018 and the previous 2011 version is the addition of a “risk-based approach” to the principles of auditing. Specifically, clause 4(g) of the guidelines for auditing management systems is, “The risk-based approach should substantively influence the planning, conducting and reporting of audits to ensure that audits are focused on matters that are significant for the audit client, and for achieving the audit program objectives.” A lot of people are unsure of what is meant by a risk-based approach. Still, the key to understanding this is to focus on the definition of risk. From a product perspective, the risk is the “combination of the probability of occurrence of harm and the severity of that harm.” From a process perspective, the risk is the “effect of uncertainty on an expected result” (ISO 9001:2015, clause 3.09). Therefore, auditors should emphasize medical devices with the highest severity of harm and devices with a high probability of hazards or hazardous situations. When an auditor focuses on a process rather than a specific medical device, auditors should emphasize any processes that are not under control and any recent process changes.

What is risk-based auditing?

Risk-based auditing considers the risks of failing to achieve audit objectives and the opportunities created by choosing various audit methods and strategies. For example, a desktop audit of procedures might be appropriate if you are conducting your first internal audit for a new quality system. Alternatively, a desktop audit would be a waste of time if you are auditing a mature quality system where very few changes to procedures have been made in the past year. Using the element approach to auditing is unlikely to add much value. Audits are meant to be a sampling. Therefore, you should focus on areas of importance where previous nonconformities were identified, any new products or processes, and anything that changed significantly.

Auditor selection should also be risk-based

Suppose you are conducting a supplier audit as part of your initial supplier qualification for a critical component supplier or contract manufacturer. In that case, you should consider doing a team audit with a multi-disciplinary team. This is a risk-based approach to the supplier qualification process, which ensures that subject matter experts evaluate each process instead of auditors with a general quality assurance background. This approach also forces more of your personnel to introduce themselves to the new supplier, and the audit will develop more reliable communication channels between your two companies. Alternatively, if you are conducting a routine internal audit of a production process, you might select a new lead auditor to conduct the audit. You don’t expect any significant findings in a routine internal audit of an established production process. In your role as an audit program manager, you need to match the new lead auditor to a process that will force them to look at all aspects of the process approach to auditing. Specifically, process validation, calibration, maintenance, and process monitoring may not apply to other administrative process areas, such as purchasing.

Risk-based auditing should influence your auditing schedule

The frequency of auditing suppliers and internal process areas should reflect the associated risks. Therefore, when you create or update your auditing schedule, you should consider the risk level of the products being audited and the process being audited. Production processes with a moderate or high level of non-conforming products may need to be audited more than once yearly. Still, a supplier with an excellent track record of extremely high quality and on-time delivery may be audited in alternating years. If you previously scheduled a remote audit, you may want to alternate to conducting an on-site audit the next time.

The duration of your audits should not always be the same either. Suppose one production process makes one product in low volume, and another production process makes multiple products in high volume. In that case, you should not schedule a two-hour internal audit for both processes every year. The low-volume production process may only need a one-hour audit once per year. In contrast, the high-volume process may require a four-hour internal audit or multiple annual audits.

Risk-based auditing applied to remote supplier auditing

The risk-based auditing approach was added to ISO 19011:2018 as the seventh principle of auditing. This represents the most significant change to that standard, but how does it apply to remote auditing? Despite the opportunities created by remote auditing, there are also risks associated with auditing suppliers remotely. People worry about auditees hiding hazardous situations or unacceptable environmental conditions such as filth or disrepair. However, unacceptable cleanliness and maintenance practices don’t happen overnight. Therefore, you should expect a clean and well-maintained facility to remain that way. One approach is to alternate between remote and on-site audits to verify the overall condition of a supplier’s facility. Therefore, the risk of auditees hiding objective evidence is more an issue of trust than a highly probable occurrence.

The more probable risks associated with remote auditing are related to the potential lack of availability of records. This is especially important for paper-based quality systems. Most people try to address this risk by scanning paper documents and records, but scanning documents have limited value. Scanning paper documents is more efficiently performed in a large batch by an automated or semi-automated process. Also, auditors and inspectors typically focus on the most recent records, and auditors and inspectors rarely sample 100% of the records. Therefore, the best risk controls include the following:

Ask a guide to send a digital picture of the record.
Use a tripod-mounted HD webcam focused on a music stand or similar surface.
Ask the auditee to read the document while you take notes.

In our experience, you will probably rely on all three risk controls, but it is unlikely to delay the audit. However, in response to the limited physical access to medical device facilities and personnel, certification bodies are sending out questionnaires to assess the risk of being unable to achieve audit objectives or cover the required scope of surveillance and recertification audits. As the audit program manager, you can reduce these risks by working with supply chain managers to develop new supplier questionnaires that specifically ask questions about the capability of supporting audits remotely. In particular, it would be essential to obtain facility maps to identify areas with inadequate cellular coverage and identify records that are only available in hardcopy format.

ISO 19011 – Do you need this quality system auditing standard? Read More »

Jan 10 2023

Why modernize 21 CFR 820 to ISO 13485?

Leave a Comment / FDA, Human Factors, ISO 13485:2016, Post-Market Surveillance, Quality Management System, Software Verification and Validation / By Robert Packard

The FDA patches the regulations with guidance documents, but there is a desperate need to modernize 21 CFR 820 to ISO 13485.

FDA Proposed Amendment to 21 CFR 820

On February 23, 2022, the FDA published a proposed rule for medical device quality system regulation amendments. The FDA planned to implement amended regulations within 12 months, but the consensus of the device industry is that a transition of several years would be necessary. In the proposed rule, the FDA justifies the need for amended regulations based on the “redundancy of effort to comply with two substantially similar requirements,” creating inefficiencies. In public presentations, the FDA’s supporting arguments for the proposed quality system rule change rely heavily upon comparing similarities between 21 CFR 820 and ISO 13485. However, the comparison table provided is quite vague (see the table from page 2 of the FDA’s presentation reproduced below). The FDA also provided estimates of projected cost savings resulting from the proposed rule. What is completely absent from the discussion of the proposed rule is any mention of the need to modernize 21 CFR 820.

Are the requirements “substantively similar”?

The above table provided by the FDA claims that the requirements of 21 CFR 820 are substantively similar to the requirements of ISO 13485. However, there are some aspects of ISO 13485 that will modernize 21 CFR 820. The areas of impact are 1) software, 2) risk management, 3) human factors or usability engineering, and 4) post-market surveillance. The paragraphs below identify the applicable clauses of ISO 13485 where each of the four areas are covered.

Modernize 21 CFR 820 to include software and software security

Despite the limited proliferation of software in medical devices during the 1990s, 21 CFR 820 includes seven references to software. However, there are some Clauses of ISO 13485 that reference software that are not covered in the QSR. Modernizing 21 CFR 820 to reference ISO 13485 will incorporate these additional areas of applicability. Clause 4.1.6 includes a requirement for the validation of quality system software. Clause 7.6 includes a requirement for the validation of software used to manage calibrated devices used for monitoring and measurement. Clause 7.3 includes a requirement for validation of software embedded in devices, but that requirement was already included in 21 CFR 820.30. The FDA can modernize 21 CFR 820 further by defining Software as a Medical Device (SaMD), referencing IEC 62304 for management of the software development lifecycle, referencing IEC/TR 80002-1 for hazard analysis of software, referencing AAMI TIR57 for cybersecurity, and referencing ISO 27001 for network security. Currently, the FDA strategy is to implement guidance documents for cybersecurity and software validation requirements, but ISO 13485 only references IEC 62304. The only aspect of 21 CFR 820 that appears to be adequate with regard to software is the validation of software used for automation in 21 CFR 820.75. This requirement is similar to Clause 7.5.6 (i.e., validation of processes for production and service provisions).

Does 21 CFR 820 adequately cover risk management?

The FDA already recognizes ISO 14971:2019 as the standard for the risk management of medical devices. However, the risk is only mentioned once in 21 CFR 820. In order to modernize 21 CFR 820, it will be necessary for the FDA to identify how risk should be integrated throughout the quality system requirements. The FDA recently conducted two webinars related to the risk management of medical devices, but implementing a risk-based approach to quality systems is a struggle for companies that already have ISO 13485 certification. Therefore, a guidance document with examples of how to implement a risk-based approach to quality system implementation would be very helpful to the medical device industry.

Modernize 21 CFR 820 to include Human Factors and Usability Engineering

ISO 13485 references IEC 62366-1 as the applicable standard for usability engineering requirements, but there is no similar requirement found in 21 CFR 820. Therefore, human factors are an area where 21 CFR 820 needs to be modernized. The FDA has released guidance documents for the human factors content to be included in a 510k pre-market notification, but the guidance was released in 2016 and the guidance does not reflect the FDA’s current thoughts on human factors/usability engineering best practices. The FDA recently released a draft guidance for the format and content of human factors testing in a pre-market 510k submission, but that document is not a final guidance document and there is no mention of human factors, usability engineering, or even use errors in 21 CFR 820. Device manufacturers should be creating work instructions for use-related risk analysis (URRA) and fault-tree analysis to estimate the risks associated with use errors as identified in the draft guidance. These work instructions will also need to be linked with the design and development process and the post-market surveillance process.

Modernize 21 CFR 820 to include Post-Market Surveillance

ISO/TR 20416:2020 is a new standard specific to post-market surveillance, but it is not recognized by the FDA. There is also no section of 21 CFR 820 that includes a post-market surveillance requirement. The FDA QSR focuses on reactive elements such as:

21 CFR 820.100 – CAPA
21 CFR 820.198 – Complaint Handling
21 CFR 803 – Medical Device Reporting
21 CFR 820.200 – Servicing
21 CFR 820.250 – Statistical Techniques

The FDA does occasionally require 522 Post-Market Surveillance Studies for devices that demonstrate risks that require post-market safety studies. In addition, most Class 3 devices are required to conduct post-approval studies (PAS). For Class 3 devices, the FDA requires the submitter to provide a plan for a post-market study. Once the study plan is accepted by the FDA, the manufacturer must report on the progress of the study. Upon completion of the study, most manufacturers are not required to continue PMS.

How will the FDA enforce compliance with ISO 13485?

It is not clear how the FDA would enforce compliance with Clause 8.2.1 in ISO 13485 because there is no substantively equivalent requirement in the current 21 CFR 820 regulations. The QSR is 26 years old, and the regulation does not mention cybersecurity, human factors, or post-market surveillance. Risk is only mentioned once by the regulation, and software is only mentioned seven times. The FDA has “patched” the regulations through guidance documents, but there is a desperate need for new regulations that include critical elements. The transition of quality system requirements for the USA from 21 CFR 820 to ISO 13485:2016 will force regulators to establish policies for compliance with all of the quality system elements that are not in 21 CFR 820.

Companies that do not already have ISO 13485 certification should be proactive by 1) updating their quality system to comply with the ISO 13485 standard and 2) adopting the best practices outlined in the following related standards:

AAMI/TIR57:2016 – Principles For Medical Device Security – Risk Management
IEC 62366-1:2015 – Medical devices — Part 1: Application of usability engineering to medical devices
ISO/TR 20416:2020 – Medical devices — Post-market surveillance for manufacturers
ISO 14971:2019 – Medical Devices – Application Of Risk Management To Medical Devices
IEC 62304:2015 – Medical Device Software – Software Life Cycle Processes
ISO/TR 80002-1:2009 – Medical device software — Part 1: Guidance on the application of ISO 14971 to medical device software
ISO/TR 80002-2:2017 – Medical device software — Part 2: Validation of software for medical device quality systems

What is the potential impact of the US FDA requiring software, risk management, cybersecurity, human factors, and post-market surveillance as part of a medical device company’s quality system?

Why modernize 21 CFR 820 to ISO 13485? Read More »

Nov 1 2022

Auditing Risk Management Files

3 Comments / Auditing, CE Marking, ISO 14971:2019 (Risk Management), ISO Auditing, Remote Auditing / By Robert Packard / Audit, ISO 14971

What do you look at and look for when you are auditing risk management files to ISO 14971 and the new Regulation (EU) 2017/745?

Your cart is empty

Next week, November 15th @ Noon EST, you will have the opportunity to watch a live webinar teaching you what to look at and what to look for when you are auditing risk management files to Regulation (EU) 2017/745 and ISO 14971. Risk Management Files are one of the essential requirements of technical documentation required for CE Marking of medical devices. Most quality system auditors are trained on how to audit to ISO 13485:2016 (or an earlier version of that standard), but very few quality system auditors have the training necessary to audit risk management files.

Why you are not qualified to audit risk management files

Being a qualified lead auditor is not enough to audit the risk management process. When you are auditing a risk management file, you need risk management training and lead auditor training. To audit the risk management process, you will also need training on applicable guidance documents (i.e., ISO/TR 24971:2020) and applicable regulations (i.e., Regulation 2017/745 and/or Regulation 2017/746). There may also be device-specific guidance documents that specify known risks and risk controls that are considered state-of-the-art.

Creating an audit agenda

Once you have scheduled an audit of risk management files, and assigned a lead auditor, then the lead auditor needs to create an audit agenda. The audit can be a desktop audit that is performed remotely, or it can be an on-site audit. Regardless of the approach, the audit should include interviewing participants in the risk management process documented in the risk management file. As a rule of thumb, I expect a minimum of 30 minutes to be spent interviewing the process owner and one or more other participants. Then I spend an additional 60 minutes of auditing time reviewing documents and records.

Your audit agenda should specify the following items at a minimum:

the method of auditing to be used,
date(s) of the audit,
the duration of the audit,
the location of the audit, and
the auditing criteria.

The auditor(s) and the auditee participants should be identified in the audit agenda. Finally, you should specify which documents and records are required for audit preparation. These documents will be used to help identify audit checklist questions and to determine a sampling plan for the audit. At a minimum, you will need a copy of the risk management procedure and a list of the risk management files that are available to audit. You may also want to request the audit plan for each of those risk management files.

What did you look at and look for during your risk management audit?

When you audit the risk management process, you could take any of the following approaches or a combination of more than one. You could audit the process according to the risk management procedure. You could audit the process according to the risk management plan(s) for each risk management file. You could audit using the process approach to auditing. Finally, you could audit in accordance with specific requirements in the ISO 14971:2019 standard and applicable regulations (i.e., Regulation 2017/745). Regardless of which approach you take, your audit notes and the audit report should identify which documents and records you sampled and what you looked for in each document. Providing only a list of the documents is not enough detail.

Creating an auditing checklist for risk management files

Auditors with limited experience are taught to create an audit checklist by creating a table that includes each of the requirements of the audit criteria. For a risk management file, this would include a list of each of the requirements in ISO 14971 for a risk management file (i.e., Clause 9???). However, this approach is more like the approach that you should be using for a gap analysis. The better approach for creating an audit checklist for risk management files is to start by creating a turtle diagram. In the “process inputs” section (i.e., step 2 of 7), you would add questions derived from your review of the risk management plan(s). In the “process outputs” section (i.e., step 3 of 7), you would add questions specific to the risk management report and other records required in a risk management file. In the “with whom” section (i.e., step 5 of 7), you would add questions related to training and competency. You might also identify additional people involved in the risk management process, other than the process owner, to interview as a follow-up trail. In the “how done” section (i.e., step 6 of 7), you would add questions specific to the procedure and forms used for the risk management process. Finally, in the “metrics” section (i.e., 7 of 7), you would verify that the company is conducting risk management reviews and updating risk management documentation in accordance with the risk management procedure and individual risk management plan(s).

Audits are just samples

Just because you can generate a lot of questions for an audit checklist does not mean that you are required to address every question. Audits are intended to be a “spot check” to verify the effectiveness of a process. You should allocate your auditing resources based on the importance of a process and the results of previous audits. I recommend approximately three days for a full quality system audit, and approximately 90-minutes should be devoted to a process unless it is the design control process (i.e., Clause 7.3 of ISO 13485) which typically requires three to four hours due to the importance and complexity of the design controls process. Therefore, you should schedule approximately 30 minutes to interview people for the risk management process and approximately 60 minutes should be reserved for reviewing documents and records. With this limited amount of time, you will not be able to review every record or interview everyone that was involved in the risk management process. This is why auditors always remind auditees that an audit is just a sampling.

Which records are required in a risk management file?

The contents of a risk management file is specified in ISO 14971:2019, Clause 4.5. There are only four bullets in that section, but the preceding sentence says, “In addition to the requirements of other clauses of this document.” Therefore, your risk management file should address all of the requirements in ISO 14971:2019. What I recommend is a virtual risk management folder for each risk management file. As the auditor, you should also request a copy of the risk management policy and procedure. An example of what this would look like is provided below. The numbers in front of each subfolder correspond to the sub-clause or clause for that requirement in ISO 14971:2019.

Which records are most valuable when auditing risk management files?

As an auditor, I typically focus on three types of targets when auditing any process. First, I will sample any corrective actions implemented in response to previous audit findings. Second, will sample documents and records associated with any changes made to the process. Changes would also include any changes that were made to individual risk management files or the creation of a new risk management file. Finally, my third target for audit sampling is any item that I feel is at risk for safety or performance failures. The severity of the safety or performance failure is also considered when prioritizing audit sampling. In the context of a risk management file, I always verify that production and post-production activities are being conducted as planned. I try to verify that risk analysis documentation was reviewed for the need to update the documentation in response to complaints and adverse events.

More auditor training on risk management files

We are recording a live webinar intended to teach internal auditors and consultants how to perform a thorough audit of risk management files against the requirements of the new European Regulation (EU) 2017/745 and ISO 14971.

Auditing Risk Management Files

In this new webinar, you will learn how to conduct a process audit of risk management files. You will learn what to look at and what to look for in order to verify compliance with Regulation (EU) 2017/745 and ISO 14971:2019. The webinar will be approximately one hour in duration. Attendees will be invited to participate in the live webinar and receive a copy of the native slide deck. Anyone purchasing after the live event will receive a link to download the recording of the live event and the native slide deck.

Price: $64.50

In addition to this webinar on auditing risk management files, we also have other risk management training webinars available. The webinar on auditing risk management files will be hosted live on November 15, 2022 @ Noon EST (incorrect in the live video announcement).

Auditing Risk Management Files Read More »

Apr 26 2022

Is monitoring every procedure required?

1 Comment / ISO 13485:2016, ISO Certification, Quality Management System / By Robert Packard

Process monitoring is required but do you know whether monitoring every procedure is required by the FDA QSR or ISO 13485?

One of the elements that Medical Device Academy has incorporated into each procedure we created in our turnkey quality system is a section titled, “monitoring and measurement.” The purpose of this section is to force each process owner to identify a process metric for monitoring every procedure. In some cases, we suggest a metric that would be appropriate for most companies establishing a new quality system. In other procedures, we use the following default text:

“Enter a quality metric that you want to track for this process in accordance with ISO 13485:2016, Clause 8.2.5 and the procedure for Monitoring, Measurement, and Analysis (SYS-017).”

Where are the requirements for process monitoring in 21 CFR 820?

Some of the companies that have purchased our turnkey quality system have asked, “Is it required to monitor and measure something in every procedure?” In general, it is not a specific requirement to have a metric specified in each procedure. In fact, if your quality system is not ISO 13485 certified, there are actually only a few places where the US FDA requires monitoring. The FDA does not have a section specific to monitoring and measurement of processes, but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used:

21 CFR 820.100(a)(1) – “Analyzing processes, work operations, concessions, quality audit reports, quality records, service records, complaints, returned product, and other sources of quality data to identify existing and potential causes of nonconforming product, or other quality problems. Appropriate statistical methodology shall be employed where necessary to detect recurring quality problems;”
21 CFR 820.200(b) – “Each manufacturer shall analyze service reports with appropriate statistical methodology in accordance with § 820.100.”
21 CFR 820.250 – “(a) Where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics. (b) Sampling plans, when used, shall be written and based on a valid statistical rationale. Each manufacturer shall establish and maintain procedures to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed. These activities shall be documented.” Note: the other two instances are the title of 21 CFR 820.250.

The word “monitoring” is equally rare (i.e. 4x) in the QSR:

21 CFR 820.70(a) – “Each manufacturer shall develop, conduct, control, and monitor production processes to ensure that a device conforms to its specifications…Where process controls are needed…(2) Monitoring and control of process parameters and component and device characteristics during production.”
21 CFR 820.75(b) – “Each manufacturer shall establish and maintain procedures for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met…(2) For validated processes, the monitoring and control methods and data, the date performed, and, where appropriate, the individual(s) performing the process or the major equipment used shall be documented.”

Where are the requirements for process monitoring in ISO 13485:2016?

ISO 13485:2016 has a section specific to monitoring and measurement of processes (i.e. Clause 8.2.5). In addition, the word “monitoring” occurs 52 times in the standard and there are 60 incidents of some variant or the exact word. , but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used. There are four Clause headings that actually include the word monitoring:

Clause 7.6, Control of monitoring and measuring equipment
Clause 8.2, Monitoring and measurement
Clause 8.2.5, Monitoring and measurement of processes
Clause 8.2.6, Monitoring and measurement of product

In Clause 1, Scope, and Clause 4.1.5, the Standard states that any outsourced processes remain the responsibility of the company and must be accounted for in the quality system by monitoring, maintaining, and controlling the processes.

Monitoring of risk is included in the definition of “risk management” in the Standard (i.e. Clause 3.18).

Clause 4.1.3 states that the organization shall, “b) ensure the availability of resources and information necessary to support the operation and monitoring of these processes…d) monitor, measure as appropriate, and analyze these processes.”

Clause 4.2.3 states that the contents of the Medical Device File (i.e. MDR or TF), shall include, “d) procedures for measuring and monitoring.”

Monitoring and measurement of processes and product are required inputs to the Management Review in Clauses 5.6.2e) and f).

Clause 6.4.1 requires a procedure for monitoring the work environment if it can have an effect on product quality.

Clause 7.1 requires the company to consider including monitoring in product realization planning.

Clause 7.4.1 requires a plan for monitoring suppliers.

Clause 7.5.1 requires monitoring production and service, including the monitoring of process parameters and product characteristics.

Clause 7.5.6 requires monitoring of validated process parameters.

Clause 7.5.8 requires identification of status with regard to product monitoring and measurement (i.e. inspection status).

Clause 7.6 requires monitoring and measurement of calibrated devices and validation of any computer software used to monitor calibrated devices.

Clause 8.1 states that companies shall plan and implement monitoring and measurement of processes.

Clause 8.2 is titled, “Monitoring and measurement.”

Clause 8.2.1 requires monitoring of customer feedback.

Clause 8.2.5 requires monitoring of processes to ensure planned results are achieved.

Clause 8.2.6 requires monitoring of products to ensure product requirements have been met.

Clause 8.4 requires data analysis of monitoring data from at least six different processes:

Feedback
Conformity to product requirements
Characteristics and trends of processes and products, including opportunities for improvement
Suppliers
Audits
Service reports, as appropriate

In summary, while not every single clause that requires a procedure includes a requirement for monitoring, there are a number of processes where the requirement to monitor the process is explicitly stated.

Why do all of our procedures include the requirement for metrics?

Medical Device Academy expanded the requirement for monitoring to all procedures for five reasons:

Quality objectives must be “established at relevant functions and levels within the organization.” Therefore, establishing monitoring requirements for each procedure ensures that top management has metrics for every process and a lack of data is never an excuse for not establishing a new quality objective when improvement is needed.
If every procedure has a requirement for monitoring, then employees don’t have to remember which processes require monitoring and which processes do not explicitly require monitoring.
The process approach to auditing includes metrics of the process as one of the seven items that are included in every process turtle diagram, and therefore, including metrics for each procedure facilitates the process approach to auditing.
If a company does not have a process metric already established, it is often difficult to perform an investigation of the root cause of quality issues. If a metric is already being monitored for the process, this facilitates the investigation of the root cause and you can use the baseline monitoring data to help you establish effectiveness criteria for the corrective action.
Finally, most companies struggle to identify preventive actions as required by Clause 8.5.3, and we have found that data analysis of monitoring data is the best source of identifying new preventive actions.

What are the disadvantages when you monitor and measure something in every procedure?

The primary reason for resistance to identifying a metric for monitoring in every procedure is that it will increase the workload for the employees responsible for that process. However, monitoring of data does not always increase workload. In fact, when process data is recorded in real-time on a run chart it is often possible to identify a trend much earlier than when data is simply recorded and subjected to monitoring.

Example #1: The automatic tracking of toner in a printer tells HP when to ship you a new toner cartridge before you need it. This ensures that there is no loss in productivity because you never run out of ink or the ability to print documents.
Example #2: Companies will use project management software (e.g. Asana) to monitor labor utilization. This will help identify when a specific resource is nearing capacity. When this occurs, the project manager can add time buffers to prerequisite steps and adjust the starting date of the resource-limited tasks to an earlier starting date. This ensures that more time is available to finish the task or to take advantage of resource availability at an earlier date.
Example #3: Monitoring the revision date for procedures helps the document control process owner identify procedures that should be evaluated for the need to be revised and updated. Often this is articulated as a quality objective of reviewing and updating all procedures within 2 years. This also ensures that procedures remain current and compliant with regulatory requirements.

What are the advantages of monitoring every procedure?

The phrase “what gets measured gets managed” is a popular business philosophy that implies measuring employee activity increases the likelihood that employees will complete a task or perform it well. In contrast, if a process is not monitored, employees may assume that it is not important and the tasks may be skipped or completely forgotten. Setting quantitative goals is also sometimes integrated with economic incentives or bonuses that are granted to individuals and teams.

FDA transition from QSR to ISO 13485

The US FDA is planning its transition from 21 CFR 820 to ISO 13485 as the quality system criteria. This will force companies to make adjustments to their quality systems and increase the amount of process monitoring performed. My general advice is to work with employees that are performing tasks to identify streamlined methods for monitoring those tasks without being overly burdensome. Then you and the employees you manage can analyze the data together and identify opportunities for improvement. When you do this, experiment with manual methods using whiteboards and paper charts that are visible in public areas first. Only implement automated solutions after you have optimized the data being collected and the frequency of data collection, and remember that not every process will benefit from automated statistical process control. Sometimes the simple approach is best.

Is monitoring every procedure required? Read More »

Jul 1 2020

Why remote audit duration should never exceed 90 minutes

1 Comment / Auditing, ISO Auditing, Remote Auditing / By Robert Packard

This article explains why remote audit duration should not exceed 90 minutes and the unique opportunities created by a series of short remote audits.

Parkinson’s Law and the subject of audit duration

On November 19, 1995, Cyril Northcote Parkinson published an essay in the Economist. The title of the article was “Parkinson’s Law.” In the first sentence of the essay, Parkinson says, “It is a commonplace observation that work expands to fill the time available for its completion.” This essay refers to the observation that work is elastic concerning the demands on time when completing paperwork. When I first trained as an auditor, trainers emphasized that the most significant challenge faced by auditors is to complete an audit within the time available. An auditor’s task is to achieve the audit objectives within the time specified by the audit program manager. Time is precious, and you cannot easily extend the audit duration after scheduling the audit.

How much time is needed for a full quality system audit?

This question is a silly question to ask a consultant that works on an hourly basis. A consultant working on an hourly basis will make more money if they work more hours. Therefore, there is little incentive to underestimate the time required to complete the objectives of an audit. However, after completing hundreds of audits, I can honestly state that eight hours is not enough time to perform a full quality system audit of a medical device company’s quality system. However, I completed a full quality system audit of a small company in less than two days. I also had difficulty completing an audit of a larger company in four days. An FDA inspector typically requires four days to complete a routine inspection, even at foreign manufacturers where English is a second language, and they only need to return on the fifth day to prepare their FDA 483 observations to give to the company. Therefore, three days is typically the absolute minimum time required to complete a full quality system audit.

Does Parkinson’s Law apply to audit duration?

Parkinson’s Law certainly applies to the audit duration. If the lead auditor assigns a team member to review the CAPA process, the task is unlikely to be completed in 30 minutes, and most auditors would struggle to appear busy for more than three hours. You need enough notes to provide objective evidence of conformity for your audit report, but if you finish too quickly, then others may perceive that you were not thorough. Therefore, most auditors will begin any process audit by asking for a copy of the procedure and a log of the records available. The auditor will quickly review the procedure’s revision history to determine when the last revision was made and if there have been any significant revisions since the last audit. Next, the auditor will review the log to estimate how many records should be sampled. The auditor will then estimate how much time is needed to review the sampled records. Finally, a quick mental calculation is made to determine how much time remains for procedure review before the auditor must move on to interview the next subject matter expert.

Why are auditors always behind schedule?

An auditor begins with small, close-ended questions that are designed to put the auditee at ease. The auditor may even comment on unrelated subjects to build rapport first. Records may not be readily available, but auditors almost always have to wait for record retrieval. The request is recorded, copies are made, and the subject matter expert may need a little time to review before handing the auditor the requested record. Auditors will ask clarifying questions, and auditees will need a few moments to check their facts. Any one of these delays is insignificant by itself, but collectively there may be two-and-half minutes of delay cumulatively for each record requested if you sample five records, which represents a combined delay 12.5 minutes. If you average only seven minutes to review each record, then a sampling of five records will require 47.5 minutes. This will leave you only 12.5 minutes for introductions, review of the procedure, and conclusions. If you want to interview any of the people that investigated root-cause, then you will need more than an hour to complete your audit, and you will not finish in the one hour scheduled.

Why is it so hard to complete a full quality system audit in three days?

Most of your process audits require a few more minutes than you expected, but you will also need time to walk to the next subject matter expert, or you will be waiting for the next subject matter expert to enter the conference room. If the quality system consists of only the minimum twenty-eight required procedures, your full quality system audit will require more than 28 hours to complete. If there are additional regulatory requirements for CE Marking or ISO 13485 certification, you will need even more time to audit every process. You should also expect certain processes to require more time to properly sample records, such as technical documentation and design controls. Even the most experienced auditors struggle to review a technical file and/or design history file in less than two hours.

What happens to an auditor after auditing all day?

As a Notified Body auditor, I used to leave my home in Vermont on Sunday afternoon and drive two hours to the nearest major airport. Then I would be gone all week conducting audits. On Friday, I would drive home and arrive in the middle of the night. Each day audits would begin early in the morning, and I would complete the day after 8.5 to 9 hours of work. Jet lag, sleep deprivation, too little exercise, and constantly eating at restaurants took its toll. I would consult my Google calendar to learn what city I was in each morning, and to remember what company I was on my schedule for the day. I would purposely try to do as much walking around during the day just to keep my blood flowing and to help stay awake. I would read documents while pacing back-and-forth in conference rooms, and I would always make sure that we had to audit the most remote area of a facility after lunch to make sure that I didn’t fall asleep. I will tell stories and jokes to entertain my hosts, but it was necessary to break up the monotony of auditing quality systems seven days a week. I would make sure I drank at least six liters of water each day for health, but this also gave me an excuse to go to take frequent bathroom breaks. Somehow I managed to survive that lifestyle for more than three years. Each day my feet, legs, back, and neck were in severe pain. I had constant headaches, and I know the quality of my work gradually declined throughout each day. The most valuable lesson I learned was, you need to move frequently, or you will die.

What happens when you sit in front of a computer for eight hours?

I can sit in front of a computer longer than almost anyone I know. When I focus on work, four hours can elapse without me getting up from a chair even once. I might pick up my empty coffee mug four or five times to take a sip before I am conscious of the need to get another cup. On days where my schedule consists primarily of Zoom meetings, I may sit through as many as six consecutive meetings before I take the time to get up and go to the bathroom and get a drink of water. Clients may perceive that I have tremendous endurance, but there are negative consequences to this work pattern. My wrist becomes sore, and I need to switch my mouse pad and the style of the mouse I am using every day. I change computers, switch microphones, and take a short walk. My neck, back, and legs will hurt worse than any of the audits during my years as a Notified Body auditor. Sitting at a computer all day has resulted in mild symptoms of restless legs syndrome. Sitting at a computer continuously for the audit duration is physically exhausting and tedious. If you must complete a remote audit on a continuous eight-hour day, you can, but it is not healthy or productive. The negative health consequences and negative impact on productivity are equally applicable to auditees.

What can you do to reduce audit fatigue during a remote audit?

The most straightforward strategy for reducing fatigue is to take breaks. Instead of auditing for eight hours continuously, try auditing in two or three 90-minutes segments each day. If you are auditing someone in a different time zone, you may only be able to accommodate an audit duration of one 90-minute session per day without working through the night. Taking breaks will allow you to leave your computer, eat food, and even go to the bathroom. You can recharge your headset during a break too. You should consider taking a walk outside. It is incredible how much better you feel when you get some exercise, stretch, and experience a little natural light instead of the unnatural glow of your computer’s monitor. The person you are auditing will appreciate the breaks, but they will also enjoy the improvement in your overall demeanor. A simple smile after a 30-minute break has a tremendous positive impact.

How can we utilize breaks more effectively during remote audits?

Auditors need documents and records to review as objective evidence. The most obvious way to make use of breaks is for the auditor to give the auditee a list of documents and records to gather during the break. This will give the auditee an excuse to go and get the documents and records if they are stored in another location. The auditee might also scan records during a break. A break also gives subject matter experts time to re-familiarize themselves with the documents and records before resuming the audit. Auditees and auditors will need to recharge batteries, but the auditor might take time to convert their notes into a summary for the final audit report. The auditor might also review the audit criteria one more time before writing a nonconformity. The auditee might take advantage of the break to initiate a new CAPA and write a draft of the corrective action plan. Then when the audit resumes, the auditee can review the draft plan with the auditor to ensure that the plan is appropriate and nothing was accidentally omitted from the CAPA plan.

Why are 90 minutes a magical audit duration?

Auditing one process in a single 45-60 minute session is ok, but if you audit two processes in a single 90-minute session, you can reduce the time spend starting and stopping the audit session by half. Adding a third process to a single session will have a smaller impact, and the meeting will need to be so long that most participants will begin to lose concentration, and fatigue becomes a significant factor. Ninety minutes is not quite long enough to audit two processes effectively. Still, an auditor can request procedures in advance of the session or spend time after the session reviewing procedures. Therefore, by paying an additional 30 minutes reviewing two procedures “off-line,” the auditor can dedicate 100% of the “on-line” time to reviewing records and interviewing subject matter experts. The result is a fast-paced, 90-minute session where each subject matter expert typically is only needed for 45 minutes. Alternatively, if you are auditing more complex records like a design history file, you can spend all 90 minutes discussing that area.

Why remote audit duration should never exceed 90 minutes Read More »

May 26 2020

Remote audit resources – software and hardware tools

2 Comments / Remote Auditing / By Robert Packard

If you are planning a remote audit, you will need more remote audit resources than a webcam and web conferencing software. Matthew Walker is a significant contributor as co-author of this article.

Clause 5 of ISO 19011:2018 is titled “Managing an audit program,” and subclause 5.4.4 is specific to determining audit program resources. For conducting audits remotely, you will need remote audit resources. Almost every laptop has a built-in webcam and microphone, and that is the minimum functionality you will need to conduct a remote audit. However, adding other software and hardware technology can improve the efficiency and effectiveness of your audit team.

What remote audit resources do you need?

Remote audits are not the same as a desktop audit, because a remote audit requires remote access to more than emails containing procedures and records. Auditors need access to people and access to physical areas of your facility. This creates one of the most significant challenges for this type of audit method. Call me a Negative Nancy, but I suspect that most audit plans do not specifically include logistical preparations to support this audit method. On the surface, it seems like a simple concept. Internet access and a scanner should cover most of the needs for the auditee to survive this digital encounter. In practice, conducting a remote audit that genuinely adds value and does more than checkboxes, requires serious planning.

Let’s start with the obvious; a remote audit needs a way for the auditor and the auditee to communicate with each other. Ideally, you need more than your phone. We recommend Zoom for video conferencing, but we list several other video conferencing software applications below. Here are the features of Zoom that we typically use during a remote audit:

Video Chat – Using Zoom, two or more parties can communicate using video input from webcams. This is nice because it allows for a more visual conversation, and you can see more of the facial expressions and body language of the person you are speaking with than you can with a traditional phone call. It also allows for sign language to be used if necessary.
Screen Sharing – Screen sharing is an essential tool you will use during a remote audit because it allows you to share documents and records on your screen even if you are not the host. The more records you have electronically, the more valuable screen sharing will be during the audit. An auditor can say, “Can you show me that quality system certificate again?” or “Can you show me where Isomedix is on your approved supplier’s list?” Being able to facilitate those verification activities saves the auditee the hassle of emailing documents or uploading content to a shared folder. This ability to share your screen is also essential for an auditee to demonstrate training effectiveness and competency.
Recording – Meetings can be recorded in their entirety or sections. This allows the auditee to record the opening or closing meetings of the audit to share with others that were unable to attend. If there are questions regarding non-conformities or opportunities for improvement, a recording of the conversation ensures that the auditor has an accurate record of complex objective evidence that would slow down the audit and gives managers a perfect record to demonstrate the issue when corrective actions are initiated.
Chat Record – Zoom, and most other video conferencing software, provides a chat box that can be used to take notes. If someone runs to the bathroom, and you don’t want to forget your question, you can enter it in the chatbox. Chat boxes are especially helpful when there is a language barrier, or someone’s accent is hard to understand. Text typed in the chatbox also serves as a place to record information that may be difficult to remember if you cannot access your audit report. If a production area has too much background noise, the chat feature might be the best way to communicate important details, such as: “That information is found in section 7.5.6 of the Quality Manual; POL-001 rev A.” The chatbox can also be used to communicate a list of documents, or records in a specific date range, that you want an auditee to make available for you to review off-line. Other participants observing the audit may also be responsible for collecting those documents in real-time to ensure the audit can continue without any delay. Finally, content in the chatbox can be recorded as a text file automatically.
Tour Guide – Video chat allows auditees to bring auditors into physical places of their facility as if the auditor were there in person. Production employees can be interviewed, in person and in real-time, while the employee demonstrates processes. You can show how nonconforming materials are labeled and segregated to keep them from accidentally being used for production. When requesting this audit method in an audit agenda, the lead auditor should recommend a dedicated “camera person” with a mobile phone and selfie stick, because it is challenging to answer auditor questions and operate a video camera simultaneously. Remember, remote audit resources consist of hardware, software, and people.

My favorite remote auditing tools (hardware)

My favorite hardware resource is the Pixelbook that I am using to write this article. We write audit reports with Google Docs instead of Microsoft Word because multiple team members can simultaneously edit the same document without creating conflicting versions. We operate Zoom video web conferencing software to speak with auditees and clients, but we use the Pixelbook to type our notes and audit reports. The Pixelbook is lightning fast, and it is a little smaller, so there is just enough room on your desk next to a laptop. The most significant advantage of using Google Docs is realized when you are the lead auditor of an audit team. As a lead auditor, you can type notes in the section of the audit report that other team members are working on, to make sure that they include audit trails from other members of the audit team. This is also an extremely useful technique when you are training a new auditor, and you want to guide them without disrupting the flow of an interview with a subject matter expert.

My second favorite hardware resource is an HD webcam mounted on a flexible arm with a clamp (see picture above). The video quality is 1080p instead of the 720p that is typical of a laptop camera. The flexible arm is equally essential because you can look directly at the camera while I’m simultaneously looking at the monitor. The only thing I dislike about the webcam I am using is the audio quality. Therefore, I use a gaming headset with a microphone to record the audio, so I can hear the people I am interviewing better. Another alternative is high-quality microphone and headphones, as typically seen in use by podcasters. Even though the sound quality is ideal with a separate microphone and headphones, the cost is higher than most gaming headsets, and you will be tethered to microphone–either physically or at least virtually by the need to maintain a consistent distance between your mouth and the microphone. The more hours you spend at the computer, the more you will appreciate the ability to stand up, adjust the camera, and move your legs a little.

Finally, the last piece of essential remote auditing hardware is your mobile phone. Even with a desktop running Zoom, and a Pixelbook running Google Docs, I still need to ask audit team members questions and conduct quick internet searches. Therefore, your mobile phone is essential to keep with you, in silent mode, during your audit. If you don’t have your phone, then you need to stop sharing your screen and send a message during your audit. Your phone is much less disruptive. I use the phone to keep track of time, to set reminder alarms, and to send Slack messages with other people. You can also join a separate Zoom session on your phone, where an audit team member may need you (the lead auditor) to provide input on objective evidence or evaluation of conformity regarding specific quality system requirements. You might also want to take a quick picture of something you observe on video during the audit. If you record the Zoom session, you can always extract a still image, but taking a picture with your mobile phone is more convenient and takes less time. You can then share the image with a Google Drive folder for your remote audit and copy the image into your audit report. As they say, a picture is worth 1,000 words.

One last note on hardware: a 48” flat screen is great for virtual bike rides on your trainer (as seen in the picture above), but it’s just a little too big for a desktop monitor. It’s excellent for side-by-side viewing, but dual monitors are a better approach.

Remote Auditing Resources for Web Conferencing

Currently, we are using Zoom as our video web conferencing software. Still, we used to use GoToMeeting, and there is very little difference in the functionality of the two software platforms. One of the consequences of the COVID19 pandemic is that everyone is more familiar with web conferencing software. Here are a few other options you could consider, including Slack, which we use as a messaging tool, and we have integrated with Zoom within our team’s channel.

Remote Audit Resources for Scheduling Your Audit

Currently, we are using Calendly as the automated appointment scheduling software application for our consulting business. However, the functionality of software applications has changed dramatically in the past few years with better integration tools, such as Zappier. Therefore, don’t be surprised if we change to one of the applications listed below. These applications allow you to manage people, equipment, and conference rooms, but you can also integrate these applications with accounting business processes.

Remote Auditing Accessories

We hosted three international training workshops, and we record training videos for medical device companies every week. Therefore, we gradually accumulated all of the accessories listed below. Technology gadgets for recording videos are continually changing, and our best advice is to save your money. Instead, rely upon a mobile phone and an extra person with “the original selfie sticks” (i.e., arms). Once you complete your first remote audit, then you can think about which of the latest gadgets might make your life easier.

Selfie Sticks
Tripod
External microphones
Portable Batteries
Additional lighting

If you have any suggestions for additional hardware and software for remote auditing, please add a comment to this article so we can keep this up to date with the latest technology.

Future Articles & Webinars

Thank you for reading. This article is our third in a ten-part blog series specific to remote auditing techniques:

Remote audit opening meeting – 4 changes – May 12
Audit team communications – May 19
Remote audit resources – software and hardware tools – May 26
How to apply a risk-based auditing approach to audits and remote audits – June 2
How to make a supplier questionnaire for remote auditing – June 25
Remote audit duration less than 90 minutes – June 30
Remote auditing work instruction – July 14
Planning partial remote audits – July 21
Remote audit invitations – 4 things to remember – August 4
Training new audit team members and lead auditors – August 11

Five (5) new webinars planned on related topics:

Opening Meetings Webinar (free) – May 14, 2020
Audit team communication during a remote audit (free) – June 4, 2020
How to qualify your suppliers webinar (pre-order by June 1) – June 25, 2020
Remote auditing techniques webinar (pre-order by July 1) – July 16, 2020
MDSAP Certification Body Interviews (free) – August 6, 2020

Remote audit resources – software and hardware tools Read More »

May 19 2020

Audit team communication

4 Comments / Remote Auditing / By Robert Packard

The lack of visual cues may hinder communication between the auditor and the auditee, but software tools can enhance audit team communication.

Audit Team Communication Requirements

During the opening meeting, the lead auditor is responsible for confirming the “formal communication channels between the audit team and the auditee…[and] the auditee being kept informed of audit progress during the audit” (ISO 19011:2018, Clause 6.4.3). Typically, the audit program manager will follow the lead auditor during the audit. In that situation, audit team communication with the auditee is direct and verbal. However, if the audit team consists of multiple auditors, the lead auditor also needs to establish a method of communication between the team members and the lead auditor. Team members need to make the lead auditor aware of any potential nonconformities, but more critical information includes:

audit trails that require follow-up by auditors in other process areas
any delay experienced by team members
if an audit team member is ahead of schedule

Communication Limitations During On-Site Audits

During an on-site audit, it is not uncommon to have limited communication with the rest of the team, because the team is interviewing auditees and walking through the facility–not sitting at their computer. Sometimes your cellular signal is inadequate for texting or other messenger services such as Slack. It may also be more difficult to have private discussions between team members during an on-site audit. Usually, the audit schedule is very tight, and team discussions must wait until lunch breaks or scheduled team discussions. Unfortunately, these limitations frequently result in the follow-up of audit trails waiting until the very end of the audit, instead of addressing audit trails at more convenient times in the middle of the audit.

Communication Between Auditors During Remote Audits

During a remote audit, all of the audit team members will readily be able to exchange information by email, text, or Slack. Besides, applications like Google Docs allow multiple auditors to type in the same audit report simultaneously. Therefore, auditors can type a specific follow-up item in the section of the audit report, where another auditor will be typing their notes for the applicable audit area. For example, if one auditor is interviewing incoming inspection activities, they can type a note for the auditor that will be auditing calibration to review the calibration certificates for inspection devices used in the incoming inspection process. If an audit team leader needs more time, they can type a quick note for the lead auditor about the need for more time. The lead auditor can also quickly send a Slack message to the rest of the audit team, asking if anyone can aid the audit team member that is behind schedule. This communication is efficient, documented directly within the report, and occurs in real-time. The result is that communication between team members is more effective, and the audit is completed earlier.

Improvement of Auditor Training with Remote Auditing

When audit team members are being trained, the lead auditor must observe their auditing and provide constructive feedback. Ideally, the lead auditor will wait for a “teachable moment.” This is the moment immediately after the lead auditor-in-training makes a mistake. Telling an auditor-in-training what to do during an audit teaches the auditor little. However, if the auditor is allowed to make a mistake, such as forgetting to ask for an audit record, then the lead auditor can point out the error immediately afterward. Correcting the auditor can be as simple as adding a note in red font within the audit report in the same section where the auditor is currently typing. The auditor will see the comment and make the correction, but the auditee will not be aware of the error. This approach avoids any embarrassment to the auditor, and the auditor is more likely to remember the instruction as constructive feedback that will make them better.

Remote Auditing Can Be Easily Recorded

Auditors can learn from the constructive feedback provided by a lead auditor, but they can also learn by watching and listening to themselves if the remote audit is recorded. This is especially easy to accomplish for internal audits, but suppliers may also allow recording of certain process audits. Opening meetings, closing meetings, and common procedures such as incoming inspection usually do not include confidential information. Therefore, you should be able to obtain permission to record these portions of the audit. These recordings can be reviewed by the auditor to identify when poorly worded questions were used. Auditors-in-training can identify when they miss an opportunity to follow an audit trail, or an auditor may realize that they ask auditees certain closed-ended (i.e., yes/no) questions instead of open-ended questions that will help them gather more information from the auditee.

Audit Team Communication with Guides

In addition to the communication between the lead auditor and the audit team members, audit team members also need to communicate with their audit guides. Guides should be used to communicate messages throughout the company. For example, if the audit is behind or ahead of schedule, the guide can communicate adjustments in the timing of the agenda. If an audit team member requests records to be provided, the guide can communicate this request and make sure the records are waiting for the auditor when they return to the audit conference room. Guides also are responsible for helping the audit team navigate from one process area to another during the audit, and to make sure that the audit team observes all safety and gowning requirements during the audit. Finally, guides may also be asked to act as an observer and verify objective evidence collected by the auditor.

Shifting Role of a Guide During Remote Audits

During a remote audit, requests for records to be provided and communication of deviations from the agenda can easily be communicated by the auditor chat features in the video conference, instant messengers, or email. Therefore, you might think that a guide is unneeded. However, when audit team members request viewing another area of a facility during a remote audit, it may be necessary to provide live video images of the process areas. It isn’t easy to speak with the auditor and provide live video images. It may be dangerous to walk backward through your facility, carrying a selfie stick, and concentrating on your discussion with the auditor instead of where you are walking. Instead, the guide should focus on providing live video, and the process owner should be concentrating on providing a guided tour and answering the auditor’s questions. The guide may also be asked to record certain information in video or picture format as objective evidence.

Conclusion

Audit teams should practice using shared documents in Google Docs and Slack during the audit to facilitate real-time audit team communication. Google Docs enables everyone to write their audit notes directly into an audit report template to eliminate delays in the completion of the audit report. Using Google Docs also makes it possible for the lead auditor to observe the progress of the audit in real-time. Audit team communications of audit trails for team members to follow-up can be accomplished in real-time by just adding a note about the trail in the applicable section of the audit report. Finally, remote auditing can facilitate better training of auditors.

Audit team communication Read More »