This page has been moved…https://medicaldeviceacademy.com/root-cause-analysis/
Search Results for: root cause
The author describes four tools (Five Why Analysis, Is/Is Not Analysis, Fishbone Diagram, and Pareto Analysis) and how each one can help conduct effective root cause analysis.
Quality problems are like weeds. If you don’t pull them out by the root, they grow right back.
Training on the 4 Tools
Most companies are doomed to repeat their mistakes because the root cause of their mistakes is not fixed. Why don’t companies fix their mistakes? Because the people responsible for the corrective actions (CAPA), were not adequately trained on root cause analysis. Adequate training on root cause analysis requires three things:
- Courage to admit that your process is broken
- Learning more than one tool for analyzing problems
- Practicing the use of root cause analysis tools
If your auditor identifies a nonconformity and you disagree with the finding, then you should not accept the finding and state your case. If an inspector rejects a part, and you believe the part is acceptable, then you should allow the part to be used “as is.” In both of these cases, however, you need to be very careful. Sometimes the problem is that “acceptable” is not as well-defined as we thought. I recommend pausing a moment and reflecting on what your auditors and inspectors are saying and doing. You may realize that you caused the problem.
Once you have accepted that there is a problem, you need to learn how to analyze the problem. There are five root cause analysis tools that I recommend:
Root Cause Analysis Tool # 1 – 5 Why Analysis
A “Five Why Analysis” is not just five questions that begin with the word “why.” Taiichi Ohno is credited with institutionalizing the “Five Why Analysis” at Toyota as a tool to drill down to the root cause of a problem by asking why five times. I have read about this, used this tool, and taught this concept to students, but I learned of a critical instruction that I was missing when I read Toyota Under Fire.
In that book, Jeff Liker makes the following statement, “Toyota Business Practices dictates using the ‘Five Whys’ to get to the root cause of a problem, not the ‘Five Whos’ to find a fire the guilty party.” At the end of the book, there are lessons learned from Toyota’s experience. Lesson 2 says, “There is no value to the Five Whys if you stop when you find a problem that is outside of your control.” If your company is going to use this tool, it is important that the responsible person is the one performing the five why analysis, and asks why they didn’t take into account forces that are out of their control.
Root Cause Analysis Tool # 2 – Is/Is Not Analysis
The next tool was presented to me at an AAMI course that I attended on CAPA. One of the instructors was from Pathwise, and he explained the “Pathwise Process” to us for problem-solving. A few years later, I learned that this tool is called the “Is/Is Not Analysis.” This tool is intended to be used when you are having trouble identifying the source of a problem. This method involves asking where the problem is occurring as a potential clue to the reason for the problem. For example, if the problem only occurs on one machine, you can rule out a lot of possible factors and focus on the few that are machine-specific.
The reverse approach is also used to help identify the cause. You can ask where the problem is not occurring. This approach may also lead you to possible solutions to your problem. For example, if the problem never occurs on the first or second shift, you should focus on the processes and the people that work on the third shift to locate the cause. The “Is/Is Not Analysis” is seldom used alone, but it may be the first step toward locating the cause of a quality problem.
Root Cause Analysis Tool # 3 – Fishbone Diagram
This name comes from the shape of the diagram. Other names for this diagram are the “Cause and Effect” or “Ishikawa” diagram. If a problem is occurring in low frequency and has always existed, this might not be your first tool. However, I typically start with this tool when I am doing an investigation of nonconforming product—especially when rejects suddenly appear.
If you are baffled about the cause of a problem, brainstorming the possible causes in a group sometimes works. However, I like to organize and categorize the ideas from a brainstorming session into the “6Ms” of the Fishbone Diagram.
Root Cause Analysis Tool # 4 – Pareto Analysis
The fourth root cause analysis tool is the Pareto Analysis named after Antoine Pareto. This tool is also a philosophy that was the subject of a book called The 80/20 Principle: The Secret to Achieving More with Less. The Pareto Analysis is used to organize a large number of nonconformities and prioritize the quality problems based upon the frequency of occurrence. The Pareto Chart presents each challenge in descending order from the highest rate to the lowest frequency. After you perform your Pareto Analysis, you should open a CAPA for the #1 problem, and then open a CAPA for the #2 problem. If you get to #3, consider yourself lucky to have the time and resources for it. We have an example of a Pareto Chart in our article on FDA 483 inspection observations from 2013.
Additional Training Resources
If you are interested in learning more about root cause analysis and practicing these techniques, please register for the Medical Device Academy’s Risk-Based CAPA training.
90% of usability testing submitted to the FDA is unacceptable and the root cause is simply a failure to understand the human factors process.
Human factors process inadequate?
If you submitted no usability testing to the FDA in your 510(k) submission, it would be obvious why the FDA reviewer identified usability as a major deficiency. However, you spent tens of thousands of dollars on usability testing that delayed the 510(k) submission by six months. Despite all of the time and money your company invested in the human factors process, it appears that you need to start over and repeat the entire process again. The CEO is furious, and he wants you to show him where in the 49-page FDA guidance it says that you have to do things differently.
Benefits from the human factors process
- Use errors result in serious injuries and death
- Easy-to-use products sell
- You will prevent delays in regulatory approval
Why was your rationale for no usability testing rejected?
Unlike CE Marking technical files, the FDA does not require a usability engineering file for all products. Instead, the FDA determines if usability testing is required based on a comparison of your device’s user interface and a competitor’s user interface (i.e. predicate device user interface). If the user interface is identical, then usability testing may not be required. Instead, your company should be able to write a rationale for not doing usability testing based on equivalence with the predicate device. If there are differences in your user interface, you will need to provide use-related risk analysis (URRA), identify critical tasks, implement risk controls, and provide verification testing to demonstrate the effectiveness of the risk controls. Even if your device is “easier to use” or “simpler”, you still need to provide the documentation to support this claim in your submission. The FDA also does not allow comparative claims in your marketing for 510(k) cleared devices. Comparative claims require the support of clinical data.
What is the 10-step human factors process?
- Define human factors for your device or IVD
- Identify use errors
- Conduct a URRA
- Perform a critical task analysis
- Conduct a risk control option analysis
- Conduct formative usability testing
- Implement risk controls
- Conduct summative usability testing
- Prepare HFE/UE documentation
- Collect post-market surveillance data specific to use errors
There is a YouTube video describing these 10 steps at the bottom of this blog posting.
Why is formative testing needed?
- Observational study to identify unforeseen use errors
- Observational study to evaluate risk control options
- What are the other types of studies?
- Development of indications for use
- Development of training materials
Why is the human factors process crazy expensive to outsource?
- Human factors consultants need time to learn about your device
- Consultants are more conservative because they cannot afford to fail
- Justifying your choice of risk controls is difficult because you started too late
- Your instructions for use (IFU) are inadequate
- Consultants need to explain the human factors process to you
- Recruiting subjects is marketing (which may not be their expertise)
- You are paying for infrastructure (specialized testing facilities)
- This is a team effort that requires many consulting hours collectively
Why was your Usability Engineering File refused?
- Your company provided an application failure modes and effects analysis (aFMEA) to support your justification that residual risks are acceptable. The FDA guidance suggests using risk analysis tools such as an FMEA or fault-tree analysis, but deficiency letters from FDA reviewers recommend a use-related risk analysis (URRA) format that is totally different.
The primary problem with using an FMEA or Fault-Tree risk analysis tool is that these tools involve the estimation of the severity of harm and the probability of occurrence of harm, while the FDA does not feel it is appropriate to estimate the probability of occurrence of harm. Instead, the FDA instructs companies to assume that use errors will occur and to implement risk controls to mitigate those risks (see URRA example above). Although “mitigation” is unlikely, and use risks will only be reduced, this is the approach the FDA wants companies to use. In addition, the FDA expects your company to provide traceability of risk control implementation to each use-related risk you identified and the FDA expects documentation of verification testing (i.e. usability testing) that shows your risk controls are effective. Finally, the FDA (and ISO 14971, Clause 10) expects you to collect and perform a trend analysis of use errors. Any use errors that are reported should be evaluated for the need to implement additional corrective actions to prevent future use errors. Blaming “user error” is not an acceptable approach.
- You provided risk analysis and human factors testing in your 510(k) submission, but the FDA reviewer said you need to identify critical tasks and provide traceability to each critical task in your summative validation report. – Critical tasks are specifically mentioned in section 3.2 of the FDA guidance on applying human factors and usability engineering–and a total of 49 times throughout the guidance. However, “critical tasks” are not mentioned even once in ISO 14971:2019 or ISO/TR 24971:2020. The term “critical tasks” is not even found in IEC 62366-1:2015. There is mention of “tasks”, and “task” is a formal definition (i.e. Definition 3.14, “Task – one or more USER interactions with a MEDICAL DEVICE to achieve a desired result”). Therefore, companies that are familiar with the ISO Standards and CE Marking process frequently need training on the FDA requirements for the human factors process. After receiving training, then your company will be prepared to modify your usability engineering file documentation to comply with the FDA requirements for human factors.
- You completed a summative validation protocol, but the FDA disagrees with your definition of user groups. – Each user has a different level of experience, training, and competency. Therefore, if you define the intended user population too broadly (e.g. healthcare practitioners), the FDA may not accept your summative usability testing. This is the reason that the human factors process begins with defining the human factors for your IVD or device. Radiologists, for example, have the following training pathway:
- graduate from medical school;
- complete an internship;
- pass state licensing exam;
- complete a residency in radiology;
- become board certified; and
- complete an optional fellowship.
Therefore, if you are developing imaging software, you need to make sure your user group includes radiologists that cover the entire range of competencies. In addition, most radiology images are taken by radiology technicians and then reviewed by the radiologist. Therefore, radiology technicians should be considered a completely different user group due to the differences in experience, training, and competency when compared to a radiologist. This simple example doubles the number of users needed because you have two user groups instead of one.
- You evaluated 15 users, but the FDA reviewer is asking you to evaluate a larger number of users based on a special controls guidance document. – The FDA guidance on human factors testing specifies a minimum of 15 users for each user group–not a minimum of 15 users. Therefore, for a device that is for Rx-only and OTC use, you will have at least two user groups that need to be evaluated independently. In addition, some devices have special controls guidance documents that specify usability testing requirements. For example, an OTC blood glucose meter must pass a 350-person lay-user study. Covid-19 self-tests are expected to pass a 30-person lay-user study as another example.
- Your usability study was conducted in Australia, but the FDA insists that your usability study must be repeated in the USA. – Most people think of language being the primary difference between two countries, and therefore the author of a study protocol may not perceive any difference between the USA and Australia, Ireland, Canada, or the UK. However, this lack of ability to identify differences between cultural norms shows our own ignorance of cultural differences. International travelers learn quickly about the differences in the interface used for electrical outlets between the USA and other countries. There are also more subtle differences between cultures, such as in which direction do you toggle a light switch to turn on a light, up or down? For devices that are used in a hospital environment, it is critical to understand how your device will interact with other devices and how different hospital protocols might impact human factors.
- The FDA reviewer indicated that your usability engineering file does not assess the ability of laypersons to self-select whether your OTC device is appropriate for them. – Devices and IVD devices may have contraindications or indications for use that are specific to an intended patient population or intended user population. In these cases, the user of the device or IVD needs to be able to “self-select” as included or excluded from use. The ability to self-select should be assessed as part of any OTC usability study. The ability to identify suitable and unsuitable patients for treatment is also a common criterion for a usability study involving prescription devices where a physician is the subject of the study.
- The FDA reviewer indicated that you did not provide raw data collected by the study moderator. – Data collected during a human factors study is usually subjective in nature, and the FDA may want to conduct its own review and analysis of your data. Therefore, you cannot provide only a testing report that summarizes the results of your study. You must also provide the raw data for the study. It is permitted to provide the data in a tabular format that has been transcribed from paper case report forms or was recorded electronically. You should also consider scanning any paper forms for permanent retention or retaining the paper forms in case there is any question of accuracy in the transcription of the data collected. Finally, it is best practice to record videos of the study participants performing each task and answering interview questions. This will help in filling any gaps in the notes recorded by the moderator, and the recording provides additional objective evidence of the study results.
- The FDA reviewer indicated that your study is not valid, because the training provided by moderators was not scripted and training decay was not considered in the design of the study. – Summative usability testing requires that users complete all of the critical tasks identified in your critical task analysis without assistance. It is permitted to provide training to the user prior to conducting the study if the device or IVD is for prescription use and healthcare practitioners are responsible for providing instruction to the user. However, any training provided must be scripted in advance and approved as part of the summative usability testing protocol. This ensures that every subject in the study receives consistent training. Unfortunately, the FDA may still not be satisfied with the design of your study if you do not allow sufficient time to pass between the time that training is provided to the user and when the subject uses the device or IVD for the first time. In general, one hour is the minimum amount of time that should pass between providing user training and when the device or IVD is used for the first time. This is referred to as “training decay” and the duration of time between your scripted training and the user performing critical tasks for the first time should be specified in your summative usability protocol. One solution to address both issues is to provide a video of the instructions to each subject 24-hours in advance of participation in the study.
Additional resources for the human factors process and usability testing
What is ISO 13485?
ISO 13485 is an international standard for quality management systems that is specific to the medical device industry. ISO 13485:2016 is the most recent version of the standard, and it has become the blueprint for medical device company quality systems globally. If your company wants to design, manufacture, or distribute medical devices you should consider becoming ISO 13485 certified.
Do you have to purchase a copy of ISO 13485?
Yes, you need to maintain a copy of the ISO 13485 standard as a “document of external origin.” This is needed for reference when you are making updates to procedures in your quality system. If you are looking for the best place to purchase a copy of the ISO 13485:2016 standard, we recommend the Estonian Centre for Standardisation and Accreditation. If you purchase a copy, we recommend selecting the option for a multi-user license so the standard can be used by more than one person in your company and printed. The only difference between the EN ISO version and the International ISO version is that the EN ISO version includes harmonization Annex ZA for compliance with the EU MDR and Annex ZB for compliance with the EU IVDR. This version is also referred to as A11:2021. Here’s a copy of the text from the beginning of the Standard:
“This Estonian standard EVS-EN ISO 13485:2016/A11:2021 consists of the English text of the European standard EN ISO 13485:2016/A11:2021. This standard has been endorsed with a notification published in the official bulletin of the Estonian Centre for Standardisation and Accreditation. Date of Availability of the European standard is 08.09.2021. The standard is available from the Estonian Centre for Standardisation and Accreditation.”
Medical Device Academy’s experience with ISO 13485 training
Rob Packard created his first quality system in the Spring of 2004. In October 2009, after successfully managing quality systems for three different medical device manufacturers, Rob joined BSI as a Lead Auditor and instructor. In April 2010, he purchased the 13485cert.com URL and he began to help companies implement quality systems as a consultant (while continuing to audit and train 140 days per year for BSI). In 2011 his medical device blog postings began as a way to help medical device companies. In 2012, Rob began building a library of quality system procedures for a turn-key quality system and selling the procedures from the Medical Device Academy website. Dozens and dozens of consulting clients have successfully achieved ISO 13485 certification with Medical Device Academy’s turnkey quality system procedures, and hundreds of quality systems were audited and/or improved. This ISO 13485 training webinar is also included as part of our turnkey quality system.
Projected Changes for 2023
On February 23, 2022, the FDA published a proposed rule for medical device quality system regulation amendments. The FDA planned to implement amended regulations within 12 months, but the consensus of the device industry is that a transition of several years would be necessary. In the proposed rule, the FDA justifies the need for amended regulations based on the “redundancy of effort to comply with two substantially similar requirements,” creating inefficiencies. The FDA also provided estimates of projected cost savings resulting from the proposed rule. What is completely absent from the proposed rule is any mention of the need for modernization of device regulations.
The QSR is 26 years old, and the regulation does not mention cybersecurity, human factors, or post-market surveillance. Risk is only mentioned once by the regulation, and software is only mentioned seven times. The FDA has “patched” the regulations with guidance documents, but there is a desperate need for new regulations that include critical elements. The FDA has “patched” the regulations through guidance documents, but there is a desperate need for new regulations that include critical elements. The transition of quality system requirements for the USA from 21 CFR 820 to ISO 13485:2016 will force regulators to establish policies for compliance with each of these quality system elements. Companies that do not already have ISO 13485 certification should be proactive by 1) updating their quality system to comply with the standard and 2) adopting the best practices outlined in the following related standards:
- AAMI/TIR57:2016 – Principles For Medical Device Security – Risk Management
- IEC 62366-1:2015 – Medical devices — Part 1: Application of usability engineering to medical devices
- ISO/TR 20416:2020 – Medical devices — Post-market surveillance for manufacturers
- ISO 14971:2019 – Medical Devices – Application Of Risk Management To Medical Devices
- IEC 62304:2015 – Medical Device Software – Software Life Cycle Processes
- ISO/TR 80002-1:2009 – Medical device software — Part 1: Guidance on the application of ISO 14971 to medical device software
- ISO/TR 80002-2:2017 – Medical device software — Part 2: Validation of software for medical device quality systems
Previous versions of the ISO 13485 webinars
This 2-part webinar has been previously recorded three different times. Our previous webinar on the 2003 version of ISO 13485 was split into two parts: Stage 1 and Stage 2. That first webinar was recorded in 2015. The webinars were updated in 2016 and again in 2018. We followed the same format, 2-part Stage 1 and Stage 2, for all of the subsequent ISO 13485 training webinars. The Stage 1 webinar focuses on the following processes:
- Management Review
- Internal Auditing
- Quality System Documentation
The Stage 2 webinar on the rest of the standard, including but not limited to:
The most recent version of ISO 13485 webinars
The webinars explaining the requirements for ISO 13485 were last updated in 2020. Anyone who purchases these webinars will receive free access to updated versions of the ISO 13485 training webinars. If you are making a new purchase of these two training webinars, the webinars are only being sold as a bundle for $258. You get:
- access to the Stage 1 webinar recorded on July 24, 2020
- access to the Stage 2 webinar recorded on July 28, 2020
- native slide decks for both webinars
This pair of ISO 13485 training webinars explain precisely what you need to do to implement a quality system compliant with ISO 13485. After you create your own plan (a free template is provided with a subscription), you can show the recording of these two webinars to your management team so they can implement your plan in the next several months. All deliveries of content will be sent via Aweber emails to confirmed subscribers.
Webinar duration & format
Webinars were hosted live via Zoom in 2020. The Stage 1 webinar was 64 minutes, and the duration of the Stage 2 webinar was 82 minutes. When you purchase this webinar bundle, you will receive a link to download both recorded webinars from our Dropbox folder. In addition, you will receive links to download the native slide deck for each webinar from Dropbox.
Purchase the ISO 13485 training bundle
Exam and Training Certificate available
There is a big difference between being ISO 13485 certified and being compliant with ISO 13485:2016, the medical devices quality management systems standard. Anyone can claim compliance with the standard. Certification, however, requires that an accredited certification body has followed the requirements of ISO 17021:2015, and they have verified that your quality system is compliant with the standard. To maintain that certification, you must maintain your quality system’s effectiveness and endure both annual surveillance audits and a re-certification audit once every three years.
Step 1 – Planning for ISO 13485 certification
There are six steps in the ISO 13485 certification process, but that does not mean there are only six tasks. The first step in every quality system is planning. Most people refer to the Deming Cycle or Plan-Do-Check-Act (PDCA) Cycle when they describe how to implement a quality system. However, when you are implementing a full quality system, you need to break the “doing” part of the PDCA cycle into many small tasks rather than one big task. You also can’t implement a quality system alone. Quality systems are not the responsibility of the quality manager alone. Implementing a quality system is the responsibility of everyone in top management.
Below you will find seven tasks listed. I did NOT identify these nine tasks as “Steps” in the ISO 13485 certification process, because these tasks are typically repeated for each process in your quality system. Most quality systems are implemented over time, and the scope of the quality system usually grows. Therefore, you are almost certain to have to perform all of the following nine tasks multiple times–even after you receive the initial ISO 13485 certification. As the saying goes, “How do you eat an elephant? One bite at a time.” Therefore, avoid the inevitable heartburn caused by trying to do too much at one time. Implement your quality system one “bite” at a time.
Task 1 – Purchase applicable standards
The first task in implementing an ISO 13485 quality system is to purchase a copy of the ISO 13485:2016 standard, such as the MDSAP Companion Document. You will also need other applicable medical device standards. Some of these standards are general standards that apply to most, if not all, medical devices, such as ISO 14971:2019 for risk management. There are also guidance documents that explain how to use these general standards, such as ISO/TR 24971:2020, and guidance on how to apply ISO 14971:2019. Finally, there are testing standards that identify testing methods and acceptance criteria for things such as biocompatibility and electrical safety. You will need to monitor these standards for new and revised versions. When these standards are updated, you will need to identify the revised standard and develop a plan for addressing the changes.
When you purchase a standard, be sure to buy an electronic version of the standard so you can search the standard for keywords efficiently. You should also consider purchasing a multi-user license for the standard because every manager in your company will need to look up information in the standard. Alternatively, you could buy a paper copy of the standard and locate the standard where everyone in your company can access it. Often I am asked what the difference is between the EN version of the standard and the ISO version of the standard. “EN” is an abbreviation meaning European Standards or “European Norms,” which is based upon the literal translation from the French (i.e., “normes”) and German (i.e. “norm”) languages. “ISO” versions are international standards. In general, the body of the standard is typically identical but harmonized EN standards for medical devices include annexes ZA, ZB, and ZC that identify any deviations from the requirements in three medical device directives (i.e., MDD, AIMD, and IVDD).
Task 2 – Identify which processes are applicable
Clause 1 of ISO 13485 is specific to the scope of a quality system. ISO 9001, the general quality system standard, allows you to “exclude” any clause from your quality system certification. However, ISO 13485 will only allow you to exclude design controls (i.e., clause 7.3). Other clauses within ISO 13485 may be identified as “non-applicable” based on the nature of your medical device or service. You must also document the reason for non-applicability in your quality manual. Typically, the following clauses are common clauses identified for non-applicability:
- Clause 4.1.6 – quality system software
- Clause 6.4 – work environment
- Clause 7.5.2 – cleanliness of the product
- Clause 7.5.3 – installation
- Clause 7.5.4 – servicing
- Clause 7.5.5 – sterile devices
- Clause 7.5.6 – process validation
- Clause 7.5.7 – sterilization validation
- Clause 220.127.116.11 – implantable devices
- Clause 7.5.10 – customer property
- Clause 8.3.4 – rework
Task 3 – Assign a process owner to each process
The third task is to assign a process owner to each of the processes in your quality system. Typically, you create a master list of each of the required processes. Usually, the assignments are made to managers in the company who may delegate some or all of a specific process. You should expect most managers to be responsible for more than one process because there are 28 required procedures in ISO 13485:2016, but most companies have fewer than ten people when they first implement a quality system.
Task 4 – Prioritize and schedule the implementation of each process
The fourth task is to identify which processes need to be created first and to schedule the implementation of procedures from first to last. You can and should build flexibility into the schedule, but some procedures are needed at the beginning. For example, you need document control, record control, and training processes to manage all of your other procedures. You also need to implement the following processes to document your Design History File (DHF): 1) design controls, 2) risk management, 3) software development (if applicable), and 4) usability. Therefore, these represent the seven procedures that most companies will implement as early as possible. Procedures such as complaint handling, medical device reporting, and advisory notice procedures are usually reserved for last. These procedures are last because they are not needed until you have a medical device in use.
Task 5 – Create forms, flowcharts, and procedures for each process
Forms create the structure for records in your quality system, and a well-designed form can reduce the need for lengthy explanations in a procedure or work instruction. Therefore, you should consider developing forms first. The form should include all required information that is specified in the applicable standard or regulations, and the cells for that information should be presented in the order that the requirements are listed in the standard. You might even consider numbering the cells of the form to provide an easy cross-reference to the corresponding section of the procedure. Once you create a form, you might consider creating a flowchart next. Flowcharts provide a visual representation of the process. You might consider including numbers in the flow chart that cross-reference to the form as well.
Once you have created a form and a flowchart, you are now ready to write your quality system procedure. Many sections are typically included in a procedure template. It is recommended that you use a template to ensure that none of the basic elements of a procedure are omitted. You might also consider adding two sections that are uncommon to a procedure: 1) a risk analysis of the procedure with the identification of risk controls to prevent risks associated with the procedure, and 2) a section for monitoring and measurement of the process to objectively measure the effectiveness of the process. These metrics are the best sources of preventive actions, and some of the metrics might be potential quality objectives to be identified by top management.
Task 6 – Perform a gap analysis of each procedure
Most companies rely upon internal audits to catch missing elements in their procedures. However, audits are intended to be a sampling rather than a 100% comprehensive assessment. Therefore, when a draft procedure is being reviewed and approved for the first time, or a major rewrite of a procedure is conducted, a thorough gap analysis should be done before the approval of the draft procedure. Matthew Walker created an article explaining how to conduct a gap analysis of procedures. In addition, Matthew has been gradually adding cross-references to ISO 13485:2016 requirements in each procedure. He is color-coding the cross-referenced clauses in blue font as well. This makes it much easier for auditors to verify that a procedure is compliant with the regulations with minimal effort. The success of these two methods has taught us the importance of conducting a gap analysis of all new procedures.
Task 7 – Train applicable personnel for each process
You are required to document the training requirements for each person or each job in your company. Documentation of training requirements may be in a job description or within a procedure. In addition to defining who should be trained, you also need to identify what type of training should be provided. We recommend recording your training to ensure that new future employees receive the same training to ensure consistency. Design controls training should be the first priority. You are also required to maintain records of the training. You must verify that the training was effective, and you need to check whether the person is competent in performing the tasks. This training may require days or weeks to complete. Therefore, you may want to start training people several weeks before your procedure is approved. Alternatively, you can swap the order of tasks and conduct training after the procedure approval. If that approach is taken, then the procedure should indicate the date the procedure becomes effective–typically 30 days after approval to allow time for training.
Task 8 – Approve the procedure
Approval of a procedure may be accomplished by signing and dating the procedure itself, while another approach is to create a document that lists all the procedures and forms being approved at one time. The second method is the method we use in our turnkey quality system. Companies can review and approve as many procedures at one time as they wish. Since this process needs to be defined to ensure that all of the procedures you implement are approved, the document control process is typically the first procedure that companies will approve in a new quality system. The second procedure generally is for the control of records. Then the next procedures implemented will typically be focused on the documentation of design controls, risk management, usability testing, and software development. The last procedures to be approved are typically complaint handling, medical device reporting, and recalls. These procedures are left for last because you don’t need them until you are selling your medical device.
Task 9 – Start using the procedure and generating records
The last task required for the implementation of a new quality system is to start using the procedures to generate records. All of the procedures will need records before the process can be verified to be effective. Records can be paper-based, or the records can be electronic. Whichever format you use for the record retention needs to be communicated to everyone in the company through your Control of Records procedure and/or within each procedure. If you include the information in each procedure, the records of each procedure should be listed in the procedure, and the location where those records are stored should be identified. Generally, there is no specific minimum number of records to have for a certification audit, but you should have at least a few records for each process that you implement.
Step 2 – Conducting your first internal audit
The purpose of the internal audit is to verify the effectiveness of the quality system and to identify nonconformities before the certification body auditor finds them. To successfully achieve this secondary objective, it is essential to have a more rigorous internal audit than you expect for the certification audit. Therefore, the internal audit should be of equal duration or longer in duration than the certification audit. The internal audit should not consist of a desktop review of procedures. Reviewing procedures should be part of gap analysis (i.e., task 6 above) that is conducted on draft procedures before they are approved. Internal audits should utilize the process approach to auditing, and the auditor should apply a risk-based approach (i.e., focus on those processes that are most likely to contribute to the nonconforming products, result in a complaint, or cause severe injuries and death).
After your internal audit, you will receive an internal audit report from the auditor. You should also expect findings from the internal auditor, and you should expect opportunities for improvement (OFI) to be identified. Experienced auditors can typically identify the root cause of a nonconformity more quickly than most process owners. Therefore, it is recommended for each process owner and subject matter expert to review nonconformities with the auditor and discuss how the nonconformity should be investigated. The root cause must be correctly identified during the CAPA process, and the effectiveness check must be objective to ensure that problems do not recur.
Step 3 – Initiating corrective actions
Corrective actions should be initiated for each internal audit finding immediately, to make sure the findings are corrected and prevented from repeat occurrence before the Stage 1 audit. It will take a minimum of 30 days to implement the most corrective actions. Depending upon the scheduling of the internal audit, there may not be sufficient time to complete the corrective actions. However, you should at least initiate a CAPA for each finding, perform an investigation of the root cause, and begin to implement corrective actions.
Also, to take corrective actions related to internal audit findings, you should look for internal audits from other sources. The diagram below shows several different sources of potential corrective and preventive actions.
Monitoring and measuring each process is the best source of preventive actions, while internal audits are typically the best source of corrective actions. Any quality problems identified during validation are also excellent sources of corrective actions because the validation can be repeated as a method of demonstrating that the corrective actions are effective. However, your ISO 13485 certification auditor will focus on non-conforming products, complaints, and services as the most critical sources of corrective actions. These three sources are prioritized because these three sources have the greatest potential for resulting in serious injury, death, or recall if corrective actions are not implemented to prevent problems from recurring.
Step 4 – Conducting your first management review
In addition to completing a full quality system audit before your stage 1 audit, you are also expected to complete at least one management review. To make sure that you have inputs for each of the 12 requirements in the ISO 13485:2016 standard, it is recommended to conduct your management review only after you have completed your full quality system audit and initiated some corrective actions. If possible, you should also conduct supplier audits for any contract manufacturers or contract sterilizers. It is recommended to use a template for that management review that is organized in the order of the required inputs to ensure that none of the necessary inputs are skipped. Quality objectives will need to be established long before the management review so that the top management team has sufficient time to gather data regarding each of the quality objectives. Also, you should consider delegating the responsibility for creating the various slides for each input to different members of top management. This will ensure that everyone invited to the meeting is engaged in the process, and it will spread the workload for meeting preparation across multiple people.
At the end of the meeting, top management will need to create a list of action items to be completed before the next management review meeting. Meeting minutes will need to be documented for the meeting, including the list of action items and each of the four required outputs of the management review process. We recommend using the notes section of a presentation slide deck to document the meeting minutes related to each slide. Then the slide deck can be converted into notes pages and saved as a PDF. The PDF notes pages will be your final meeting minutes for the management review. An example of one of these note pages is provided in the figure below.
One of the more common non-value-added findings by auditors is when an auditor issues a nonconformity because you do not have your next internal audit and your next management review scheduled–even though each may have occurred only a month prior to the Stage 1 audit. Therefore, we recommend that you document your next 12-month cycle for internal audits and schedule your next management review as action items in every management review meeting. The schedule can be adjusted if needed, but this allows top management to emphasize various areas in internal audits that may need improvement. You might even set a quality objective to conduct a minimum of three management reviews per year at the end of your first management review.
Step 5 – Stage 1, Initial ISO 13485 Certification Audit
In 2006, the ISO 17021 Standard was introduced for assessing certification bodies. This is the standard that defines how certification bodies shall go about conducting your initial certification audit, annual surveillance of your quality system, and the re-certification of your quality system. In the past, certification bodies would typically conduct a “desktop” audit of your company before the on-site visit to make sure that you have all the required procedures. However, ISO 17021 requires that certification bodies conduct a Stage 1 audit that assesses the readiness of your company before conducting a Stage 2 audit. Therefore, even if the Stage 1 audit is conducted remotely, the certification body is expected to interview process owners and sample records to verify that the quality system has been implemented. Certification body auditors will also typically verify that your company has conducted a full quality system audit and at least one management review. Finally, the auditor will usually select a process such as corrective action and preventive action (CAPA) to make sure that you are identifying problems with the quality system and taking appropriate measures to address those problems.
Your goal for the Stage 1 audit should not be perfection. Instead, your focus is to make sure that there are no “major” non-conformities. The term “major” used to have a specific definition:
- Absence of a documented procedure or process
- Release of nonconforming product
- Repeat nonconformities (not possible during Stage 1)
Under the MDSAP, the grading system for nonconformities now uses a numbering system for grading nonconformities: “Nonconformity Grading System for Regulatory Purposes and Information Exchange Study Group 3 Final Document GHTF/SG3/N19:2012.” Any nonconformity is graded on a scale of one to four, and then two potential escalation rules are applied. If any nonconformities are graded as a four or a 5, then the auditor must assess whether a five-day notice to Regulatory Authorities is required. A five-day notice is required in either of the following situations: 1) one or more findings grading of “5”; or 2) three or more findings graded as “4.” If your Stage 1 audit results in a five-day notice, then you are not ready for your Stage 2 audit. For example, a complete absence of two required procedures in clauses 6.4 through 8.5 of ISO 13485:2016 would result in two findings with a grading of “4.” This would not result in a five-day notice, but the absence of a third required procedure would result in a five-day notice.
The duration of your Stage 1 audit will be one or two days, but a 1.5-day audit is quite common for MDSAP Stage 1 audits. The reason for the 1.5-day Stage 1 audit is that it is challenging to assess readiness for Stage 2 in one day, and if the total duration of Stage 1 and Stage 2 is 5.5 days, then the Stage 2 audit could be completed in four days. The four-day audit is more convenient than a three-day audit for a two-person audit team.
After your Stage 1 audit, you will receive an audit report, and you should expect findings. You should initiate corrective actions for each finding immediately, to make sure the findings are corrected and prevented from repeat occurrence before the Stage 2 audit. The duration between the audits is typically about 4-6 weeks. That does not leave much time for you to initiate a CAPA, perform an investigation of the root cause, and implement corrective action. At a minimum, you must submit a corrective action plan for each finding to your MDSAP auditing organization (AO) within 15 calendar days of receiving the finding. For any findings graded as a “4” or higher, you will need to provide evidence of implementing the corrective action plan to the AO within 30 calendar days of receiving the finding. You are also unlikely to have enough time to conduct an effectiveness check prior to the Stage 2 audit.
Step 6 – Stage 2, Initial ISO 13485 Certification Audit
The Stage 2 initial ISO 13485 certification audit will verify that all regulatory requirements have been met for any market you plan to distribute in. The auditor will complete an MDSAP checklist that includes all of the regulatory requirements for each of the countries that recognize MDSAP: 1) the USA, 2) Canada, 3) Brazil, 4) Austria, and 5) Japan. The auditor will also sample records from every process in your quality system to verify that the procedures and processes are fully implemented. This audit will typically be at least four days in duration unless multiple auditors are working in an audit team.
The audit objectives for the Stage 2 ISO 13485 certification audit specifically include evaluating the effectiveness of your quality system in the following areas:
- Applicable regulatory requirements
- Product and process-related technologies
- Technical documentation
All procedures will be reviewed for compliance with ISO 13485:2016 and the applicable regulations. The auditor will also sample records from each process. If the auditor identifies any nonconformities during the audit, it is important to record the findings and begin planning corrective actions immediately. If you have any questions regarding the expectations for the investigation of the root cause, corrections, corrective actions, and effectiveness checks, you should ask the auditor during the audit or the closing meeting. At a minimum, you must submit a corrective action plan for each finding to your MDSAP auditing organization (AO) within 15 calendar days of receiving the finding. For any findings graded as a “4” or higher, you will need to provide evidence of implementing the corrective action plan to the AO within 30 calendar days of receiving the finding. The auditor will not be able to recommend you for ISO 13485 certification until your corrective action plans are accepted.
If you receive a finding with a grading of “5,” or three or more findings graded as “4,” then the MDSAP auditor is required to issue a five-day notification to the regulators. The auditor will also need to return to your facility for a follow-up audit to close as many findings as they can. It is not necessary to eliminate all of the findings in order to be recommended for ISO 13485 certification, but the grading of the findings must be reduced to at least a “3” before recommending the company for certification. The number of findings also determines whether the auditor recommends your company for certification.
In addition to reviewing the findings and conclusions of the audit during the closing meeting, the auditor will also review the plan for the annual surveillance and re-certification with you. Each certification cycle is three years in duration. There will be two surveillance audits of approximately one-third of the duration of the combined duration of stage 1 and stage 2 initial certification audits, and the first surveillance audit must be completed within 12 months of the initial certification audit. In the third year, there will be a re-certification audit for two-thirds of the duration of the combined duration of stage 1 and stage 2 initial certification audits. The initial ISO 13485 certificate will be issued with a three-year expiration, and the certificate is typically received about one month after the acceptance of your corrective action plan.
There are no stupid questions, and we can save you weeks of wasted time if you just ask for help. We are always looking for new ideas for blogs, webinars, and videos on our YouTube channel. If you have any general questions about obtaining ISO 13485:2016 certification, please email Rob Packard at firstname.lastname@example.org. If you have a suggestion for new ISO 13485 training materials, you can also use our “Suggestion Box.” You can also schedule an initial free consultation with Rob using his calendly link.
Your CAPA procedure is the most important SOP. It forces you to investigate quality problems and take actions to prevent nonconformity.
CAUTION: Read the story in the next few paragraphs before you implement any purchased procedure
During a recent internal audit, I noticed that the client was not meeting one of the requirements of their CAPA procedure. Specifically, the procedure indicated that all CAPA plans must be written within seven calendar days of initiating the CAPA. Despite this requirement in their procedure, the client was indicating that CAPA plans were due within 30 calendar days on their CAPA form.
This example is a minor nonconformity, but the reason why this client was not following their procedure is more interesting. The procedure was 100% compliant with FDA regulations, but the procedure did not match how the company performed the process. The procedure and the process MUST match.
This client purchased their CAPA procedure from another consultant, changed the title, and had everyone in the company “read and understand” the procedure for training.
Make sure your CAPA procedure is clear and concise
Procedures are often unclear because the author is more familiar with the process than the intended audience for the procedure. An author may abbreviate a step or skip it altogether. As an author, you should use an outline format and match your CAPA form exactly. There should be nothing extra in the procedure, and nothing left out. Medical Device Academy’s updated CAPA procedure is only six pages and the CAPA form is four pages.
CAPA procedure writing recommendations
Procedures are often unclear because the author is an expert with more experience than the intended audience. An author may abbreviate a step or skip it altogether. As an author, you should use an outline format and match your CAPA form exactly. There should be nothing extra in the procedure, and nothing left out. Before writing your own CAPA procedure, consider following these 7 steps:
- Design your CAPA form first
- Identify which steps in your process are most important and specify how these steps will be monitored (i.e., risk-based approach)
- Write a procedure that follows your CAPA form and includes instructions for monitoring and measuring your CAPA process
- Conduct group CAPA training using the draft version of your form and procedure
- Make revisions to the form and procedure to clarify steps the trainees had difficulty with
- Ask the trainees to review the revised form and procedure
- Make final revisions and route the procedure for approval
The specific order of steps is essential to creating a CAPA procedure—or any procedure. Writing a procedure that matches the form used with that procedure helps people understand the tasks within a process. Throughout the rest of this article, we describe each of the nine steps of Medical Device Academy’s CAPA procedure (SYS-024). The actual CAPA form (FRM-009) sold with SYS-024 is more complex than 9 steps, but a more complex form is needed to make sure every sub-task is documented in your CAPA records.
Review nonconformities, including complaints, to determine if a CAPA is needed (Step 1)
If I am auditing a CAPA process, and almost all the CAPAs are resulting from auditor findings, then I know the client is not adequately reviewing other sources of potential quality issues. When I took my first CAPA training course, the image below was drawn on a flip chart by Kim Trautman. I have used this image in all of my CAPA training for other people since. I think this provides a good visual representation of the most common sources of new CAPAs. Although the number of CAPAs from each source will never be equal, you should review all of these data sources for quality issues periodically.
CAPA procedure step 2 – Describe and reference the quality issue
The next step is to copy and paste your quality issue directly into the CAPA record and add a reference to the source. This step of the CAPA procedure is not a specific requirement of the ISO 13485 standard or the FDA regulations. However, describing and referencing quality issues in your CAPA record is a practical requirement. The person assigned to investigate the root cause of the quality issue needs to know what the source of the quality issue is, and when you are trying to close a complaint or audit report you will find it helpful to cross-reference the two records. For example, the CAPA might be related to the fifth nonconformity in your second internal audit report for 2022 (e.g. IA 220205).
Attribution: Icons were copied and pasted from Flaticon.com.
Step 3 – Perform a root cause analysis
Why can’t we fix our mistakes the first time? We are doomed to repeat mistakes when we fail to identify the root cause or causes. The person you assign to investigate a quality issue must be trained to perform a thorough root cause analysis. Successful root cause analysis depends upon four things:
- Courage to admit that your process is broken
- Learning more than one tool for analyzing problems
- Practicing the use of root cause analysis tools
- Sampling enough records (or testing enough product)
The common belief is that people fail to identify the root cause because they need root cause training (#2) or more practice (#3). However, most people fail because they stop sampling or testing too soon. I typically recommend that companies sample at least twice as many records as the suspected problem frequency. For example, if a complaint occurs 1% of the time, you should review 200 records before you can be sure you identified that root cause. If you are correct, you will only find the quality issue twice in the 200 records. However, a review of 200 records often reveals that the quality problem is more common than you originally estimated and there is more than one cause of device malfunction.
CAPA procedure step 4 – Do you need a new CAPA?
After you have successfully completed a root cause analysis, you now need to determine if a new CAPA is needed. If there is already a CAPA that is open for the same quality issue, you can use the existing CAPA as justification for not conducting corrective actions. In this case, you should include a cross-reference in the new CAPA record to the existing CAPA record. You should also document containment measures and corrections.
In both the existing CAPA record and your new CAPA record you should also be documenting your risk evaluation of your CAPA. In the latest update to our CAPA procedure, we changed the method of risk evaluation to match the MDSAP grading process for nonconformities. A copy of this section is provided in the image below.
In the CAPA procedure, we state that any risk score of 4 or 5 requires the implementation of a CAPA. If any of the escalation rules apply to the risk score, you should implement a CAPA regardless of the total risk score. This is our recommendation for a method of risk evaluation, but there is no standard telling you that you must do it this way. However, we believe this method of calculation is more likely to be consistent because it is based on the MDSAP grading guidelines.
If no escalation rules apply to your risk score, it may be possible to implement containment and corrections only. If your action plan includes only containment and corrections, we recommend that you monitor the quality issue as a process metric or quality objective to identify future occurrences. If you are evaluating a new CAPA that appears to have the same root cause as an existing CAPA, you may need to update the risk score of the existing CAPA to a higher number based on the escalation rules. Escalation may impact your corrective action plan, and it should certainly affect the prioritization of your existing CAPA.
Plan and document your corrective actions, including updating documentation (Step 5)
The biggest mistake you can make in this stage of the CAPA process is to spend too much time planning your corrective actions. “Take action and document it” is the essence of this step in the CAPA procedure. If you spend all of your time planning, then you will never take action. The CAPA plan can and should be edited. Therefore, if you know a procedure needs revision, start revising the procedure immediately. You can always add more corrective actions to your corrective action plan after you write the procedure, but you need to start writing. The second biggest mistake you can make during this stage is failing to document the actions you take. If you don’t document your actions, it’s a rumor, not a record.
Do your planned actions adversely affect regulatory compliance or safety and performance? (Step 6)
One of the required actions for a CAPA is to update your procedure(s) to reflect any process improvements to eliminate the root cause. When you update procedures, you need to make sure procedural changes do not create a regulatory compliance issue. We do this by inserting a cross-reference to each regulatory requirement in our procedures. The cross-reference is then color-coded and we add a symbol for people that are color blind. Symbols also facilitate electronic searches for regulatory requirements.
If corrective actions you implement involve design changes you will need to repeat design verification and design validation to make sure design changes do not impact safety and performance. If corrective actions change your manufacturing or service processes, you will need to repeat process validation to make sure that the process changes do not impact compliance with your design specifications. These recertification and revalidations steps are frequently forgotten, and they represent the biggest challenge for review and approval of design and process changes (described in the video below).
Perform an effectiveness check – Step 7 of your CAPA procedure
Most people verify CAPA effectiveness by verifying that all the actions planned were completed, but this is not a CAPA effectiveness check. An effectiveness check should use quantitative data from your investigation of the root cause as a benchmark. Then you should verify that the performance after corrective actions is implemented resulted in a decrease in the frequency of the quality problem, a decrease in the severity of the quality problem, or both. Ideally, a process re-validation was performed because validation protocols are required to include quantitative acceptance criteria for success.
Step 8 – Record your CAPA results
You are required to record each step of your CAPA procedure in a CAPA record (FRM-009). Therefore, we created a form that is organized in the order of the CAPA process, and then we wrote the CAPA procedure to match the organization of the form. The biggest mistake we see is that the CAPA owner does not update the record to include all of the details until the CAPA plan is completely implemented. This is a mistake. You should be documenting actions when they are taken. When you gather new information, and you need to update your root cause investigation or your corrective action plan, you are allowed to modify the record. You just need to have a system that allows you to keep track of revisions. This is often referred to as an “audit trail.” If you have a paper-based system, you will need to sign and date the document each time you make an addition. If you revise previous entries, you will need to revise and reprint the CAPA record, and then you will need to sign and date the revised and reprinted CAPA record. Ideally, you will have an electronic system with an audit trail, but software budgets are not infinite.
CAPA procedure step 9 – Close the CAPA record
The last step in the CAPA procedure is to close your CAPA record. As with most quality system records, the person responsible for the process should review and approve each record for closure with a signature and date. If the person assigned to the CAPA left sections incomplete or made mistakes in completing the CAPA form, the person that made the mistake should be instructed to correct the mistake, identify that they made a correction, and identify the date of the correction. If a CAPA is not effective, then a cross-reference to the new CAPA that is opened should be documented in the older CAPA record.
If you are interested in more training on CAPA, you might be interested in purchasing Medical Device Academy’s Risk-Based CAPA webinar. 99% of companies hold off on their training until a procedure is officially released as a controlled document. In my experience, however, these procedures seem to have a lot of revisions made immediately after the initial release. New users ask simple questions that identify sections of procedures that are unclear or were written out of sequence. Therefore, you should always conduct at least one training session with users prior to the final review and approval of a procedure. This will ensure that the final procedure is right the first time, and it will give those users some ownership of the new procedure.
After you train your initial group, and after you make the edits they recommended, ask those trainees to review and edit your changes to the procedure. Sometimes we don’t completely understand what someone is describing, and sometimes maybe only half-listening. Going back to those people to verify that you accurately interpreted their feedback is the most important step for ensuring that users accept your new procedure.
After you approve your new CAPA procedure, make sure everyone in your company is trained on the final version of the procedure. CAPA is a critical process (i.e., “the heart”) in your quality system. Everyone should understand it. You should also provide extra CAPA training for department managers, such as root cause analysis training because they will be responsible for implementing CAPAs assigned to their department. You can use this 7-step process for any procedure, but ensure you use it for the most important process of all—your corrective and preventive action process.
Thank you for reading to the end of this article. Spaz and I thank you for your support.
Design controls can be overwhelming, but you can learn the process using this step-by-step guide to implementing design controls.
Design Controls Implementation
You can implement design controls at any point during the development process, but the earlier you implement your design process the more useful design controls will be. The first step of implementing design controls is to create and design controls procedure. You will also need at least two of the following additional quality system procedures:
- Risk Management Procedure (SYS-010)
- Software Development and Validation (SYS-044)
- Usability Procedure (SYS-048)
- Cybersecurity Work Instruction (WI-007)
A risk management file (in accordance with ISO 14971:2019) is required for all medical devices, and usability engineering or human factors engineering (in accordance with IEC 62366-1) is required for all medical devices. The software and cybersecurity procedures listed above are only required for products with 1) software and/or firmware, and 2) wireless functionality or an access point for removable media (e.g., USB flash drive or SD card).
Step 2: Design controls training
Even though the requirement for design controls has been in place for more than 25 years, there are still far too many design teams that struggle with understanding these requirements. Medical device regulations are complex, but design controls are the most complex process in any quality system. The reason for this is that each of the seven sub-clauses represents a mini-process that is equivalent in complexity to CAPA root cause analysis. Many companies choose to create separate work instructions for each sub-clause.
Medical Device Academy’s training philosophy is to distill processes down to discrete steps that can be absorbed and implemented quickly. We use independent forms to support each step and develop training courses with practical examples, instead of writing a detailed procedure(s). The approach we teach removes complexity from your design control procedure (SYS-008). Instead, we rely upon the structure of step-by-step forms completed at each stage of the design process.
If you are interested in design control training, Rob Packard will be hosting the 3rd edition of our Design Controls Training Webinar on Friday, August 11, 2023, @ 9:30 am EDT.
Step 3: Gathering post-market surveillance data
Post-market surveillance is not currently required by the FDA in 21 CFR 820, but it is required by ISO 13485:2016 in Clause 7.3.3c) (i.e., “[Design and development inputs] shall include…applicable outputs(s) of risk management”). The FDA is expected to release the plans for the transition to ISO 13485 in FY 2024, but most companies mistakenly think that the FDA does not require consideration of post-market surveillance when they are designing new devices. This is not correct. There are three ways the FDA expects post-market surveillance to be considered when you are developing a new device:
- Complaints and adverse events associated with previous versions of the device and competitor devices should be identified as input to the risk management process for hazard identification.
- If the device incorporates software, existing vulnerabilities of the off-the-shelf software (including operating systems) should be identified as part of the cybersecurity risk assessment process.
- During the human factors process, you should search for known use errors associated with previous versions of the device and competitor devices; known use-related risks should also include any potential use errors identified during formative testing.
Even though the FDA does not currently require compliance with ISO 13485, the FDA does recognize ISO 14971:2019, and post-market surveillance is identified as an input to the risk management process in Clause 4.2 (see note 2), Clause 10.4, and Annex A.2.10.
Step 4: Creating a design plan
You are required to update your design plan as the development project progresses. Most design and development projects take a year before the company is ready to submit a 510k submission to the FDA. Therefore, don’t worry about making your first version of the plan perfect. You have a year to make lots of improvements to your design plan. At a minimum, you should be updating your design plan during each design review. One thing that is important to capture in your first version, however, is the correct regulatory pathway for your intended markets. If you aren’t sure which markets you plan to launch in, you can select one market and add more later, or you can select a few and delete one or more later. Your design plan should identify the resources needed for the development project, and you should estimate when you expect to conduct each of your design reviews.
Contents of your design plan
The requirement for design plans is stated in both Clause 7.3.1 of the ISO Standards, and Section 21 CFR 820.30b of the FDA QSR. You can make your plan as detailed as you need to, but I recommend starting simple and adding detail. Your first version of a design plan should include the following tasks:
- Identification of the regulatory pathway based on the device risk classification and applicable harmonized standards.
- Development of a risk management plan
- Approval of your design plan (1st design review)
- Initial hazard identification
- Documentation and approval of user needs and design inputs (2nd design review)
- Risk control option analysis
- Reiterative development of the product design
- Risk analysis
- Documentation and approval of design outputs implementation of risk control measures (3rd design review)
- Design verification and verification of the effectiveness of risk control measures (4th design review)
- Design validation and verification of the effectiveness of risk control measures that could not be verified with verification testing alone
- Clinical evaluation and benefit/risk analysis (5th design review)
- Development of a post-market surveillance plan with a post-market risk management plan
- Development of a draft Device Master Record/Technical File (DMR/TF) Index
- Regulatory approval (e.g., 510k clearance) and closure of the Design History File (DHF)
- Commercial release (6th and final design review)
- Review lessons learned and initiate actions to improve the design process
Step 5: Create a detailed testing plan
Your testing plan must indicate which recognized standards you plan to conform with, and any requirements that are not applicable should be identified and documented with a justification for the non-applicability. The initial version of your testing plan will be an early version of your user needs and design inputs. However, you should expect the design inputs to change several times. After you receive feedback from regulators is one time you may need to make changes to design inputs. You may also need to make changes when you fail your testing (i.e., preliminary testing, verification testing, or validation testing). If your company is following “The Lean Startup” methodology, your initial version of the design inputs will be for a minimum viable product (i.e., MVP). As you progress through your iterative development process, you will add and delete design inputs based on customer feedback and preliminary testing. Your goal should be to fail early and fail fast because you don’t want to get to your verification testing and fail. That’s why we conduct a “design freeze,” prior to starting the design verification testing and design transfer activities.
Step 6: Request a pre-submission meeting with the FDA
Design inputs need to be requirements verified through the use of a verification protocol. If you identify external standards for each design input, you will have an easier time completing the verification activities, because verification tests will be easier to identify. Some standards do not include testing requirements, and there are requirements that do not correspond to an external standard. For example, IEC 62366-1 is an international standard for usability engineering, but the standard does not include specific testing requirements. Therefore, manufacturers have to develop their own test protocol for validation of the usability engineering controls implemented. If your company is developing a novel sterilization process (e.g., UV sterilization), you will also need to develop your own verification testing protocols. In these cases, you should submit the draft protocols to the FDA (along with associated risk analysis documentation) to obtain feedback and agreement with your testing plan. The method for obtaining written feedback and agreement with a proposed testing plan is to submit a pre-submission meeting request to the FDA (i.e., PreSTAR).
Step 7: Iterative development is how design controls really work
Design controls became a legal requirement in the USA in 1996 when the FDA updated the quality system regulations. At that time, the “V-diagram” was quite new and limited to software development. Therefore, the FDA requested permission from Health Canada to reprint the “Waterfall Diagram” in the design control guidance that the FDA released. Both diagrams are models. They do not represent best practices, and they do not claim to represent how the design process is done in most companies. The primary information that is being communicated by the “Waterfall Diagram” is that user needs are validated while design inputs are verified. The diagram is not intended to communicate that the design process is linear or must proceed from user needs, to design inputs, and then to design outputs. The “V-Diagram” is meant to communicate that there are multiple levels of verification and validation testing that occur, and the development process is iterative as software bugs are identified. Both models help teach design and development concepts, but neither is meant to imply legal requirements. One of the best lessons to teach design and development teams is that this is a need to develop simple tests to screen design concepts so that design concepts can fail early and fail fast–before the design is frozen. This process is called “risk control option analysis,” and it is required in clause 7.1 of ISO 14971:2019.
Step 8: “Design Freeze”
Design outputs are drawings and specifications. Ensure you keep them updated and control the changes. When you finally approve the design, this is approval of your design outputs (i.e., selection of risk control options). The final selection of design outputs or risk control measures is often conducted as a formal design review meeting. The reason for this is that the cost of design verification is significant. There is no regulatory or legal requirement for a “design freeze.” In fact, there are many examples where changes are anticipated but the team decides to proceed with the verification testing anyway. The best practice developed by the medical device industry is to conduct a “design freeze.” The design outputs are “frozen” and no further changes are permitted. The act of freezing the design is simply intended to reduce the business risk of spending money on verification testing twice because the design outputs were changed during the testing process. If a device fails testing, it will be necessary to change the design and repeat the testing, but if every person on the design team agrees that the need for changes is remote and the company should begin testing it is less likely that changes will be made after the testing begins.
Step 9: Begin the design transfer process
Design transfer is not a single event in time. Transfer begins with the release of your first drawing or specification to purchasing and ends with the commercial release of the product. The most common example of a design transfer activity is the approval of prototype drawings as a final released drawing. This is common for molded parts. Several iterations of the plastic part might be evaluated using 3D printed parts and machined parts, but in order to consistently make the component for the target cost an injection mold is typically needed. The cost of the mold may be $40-100K, but it is difficult to change the design once the mold is built. The lead time for injection molds is often 10-14 weeks. Therefore, a design team may begin the design transfer process for molded parts prior to conducting a design freeze. Another component that may be released earlier as a final design is a printed circuit board (PCB). Electronic components such as resistors, capacitors, and integrated circuits (ICs) may be available off-the-shelf, but the raw PCB has a longer lead time and is customized for your device.
Step 10: Verification of Design Controls
Design verification testing requires pre-approved protocols and pre-defined acceptance criteria. Whenever possible, design verification protocols should be standardized instead of being project-specific. Information regarding traceability to the calibrated equipment identification and test methods should be included as a variable that is entered manually into a blank space when the protocol is executed. The philosophy behind this approach is to create a protocol once and repeat it forever. This results in a verification process that is consistent and predictable, but it also eliminates the need for review and approval of the protocol for each new project. Standardized protocols do not need to specify a vendor or dates for the testing, but you might consider documenting the vendor(s) and duration of the testing in your design inputs to help with project management and planning. You might also want to use a standardized template for the format and content of your protocol and report. The FDA provides a guidance document specifically for the report format and content for non-clinical performance testing.
Step 11: Validation of Design Controls
Design validation is required to demonstrate that the device meets the user’s and patient’s needs. User needs are typically the indications for use–including safety and performance requirements. Design validation should be more than bench testing. Ensure that animal models, simulated anatomical models, finite element analysis, and human clinical studies are considered. One purpose of design validation is to demonstrate performance for the indications for use, but validating that risk controls implemented are effective at preventing use-related risks is also important. Therefore, human factors summative validation testing is one type of design validation. Human factors testing will typically involve simulated use with the final version of the device and intended users. Validation testing usually requires side-by-side non-clinical performance testing with a predicate device for a 510k submission, while CE Marking submissions typically require human clinical data to demonstrate safety and performance.
Step 12: FDA 510k Submission
FDA pre-market notification, or 510k submission, is the most common type of regulatory approval required for medical devices in the USA. FDA submissions are usually possible to submit earlier than other countries, because the FDA does not require quality system certification or summary technical documents, and performance testing data is usually non-clinical benchtop testing. FDA 510k submissions also do not require submission of process validation for manufacturing. Therefore, most verification and validation is conducted on “production equivalents” that were made in small volume before the commercial manufacturing process is validated. The quality system and manufacturing process validation may be completed during the FDA 510k review.
Step 13: The Final Design Review
Design reviews should have defined deliverables. We recommend designing a form for documenting the design review, which identifies the deliverables for each design review. The form should also define the minimum required attendees by function. Other design review attendees should be identified as optional—rather than required reviewers and approvers. If your design review process requires too many people, this will have a long-term impact upon review and approval of design changes.
The only required design review is a final design review to approve the commercial release of your product. Do not keep the DHF open after commercial release. All changes after that point should be under production controls, and changes should be documented in the (DMR)/Technical File (TF). If device modifications require a new 510k submission, then you should create a new design project and DHF for the device modification. The new DHF might have no changes to the user needs and design inputs, but you might have minor changes (e.g., a change in the sterilization method requires testing to revised design inputs).
Step 14: FDA Registration
Within 30 days of initial product distribution in the USA, you are required to register your establishment with the FDA. Registration must be renewed annually between October 1 and December 31, and registration is required for each facility. If your company is located outside the USA, you will need an initial importer that is registered and you will need to register before you can ship the product to the USA. Non-US companies must also designate a US Agent that resides in the USA. At the time of FDA registration, your company is expected to be compliant with all regulations for the quality system, UDI, medical device reporting, and corrections/removals.
Step 15: Post-market surveillance is the design control input for the next design project
One of the required outputs of your final design review is your DMR Index. The DMR Index should perform a dual function of also meeting technical documentation requirements for other countries, such as Canada and Europe. A Technical File Index, however, includes additional documents that are not required in the USA. One of those documents is your post-market surveillance plan and the results of post-market surveillance. That post-market surveillance is an input to your design process for the next generation of products. Any use errors, software bugs, or suggestions for new functionality should be documented as post-market surveillance and considered as potential inputs to the design process for future design projects.
Step 16: Monitoring your design controls process
Audit your design controls process to identify opportunities for improvement and preventive actions. Audits should include a review of the design process metrics, and you may consider establishing quality objectives for the improvement of the design process. This last step, and the standardization of design verification protocols in step five (5), are discussed in further detail in another blog by Medical Device Academy.
This article reviews auditor shadowing as an effective auditor training technique, but we also identify five common auditor shadowing mistakes.
How do you evaluate auditor competency?
Somewhere in your procedure for quality audits, I’ll bet there is a section on auditor competency. Most companies require that the auditor has completed either a course for an internal auditor or a lead auditor course. If the course had an exam, you might even have evidence of training effectiveness. Demonstrating training competence is much more challenging. One way is to review internal audit reports, but writing reports is part of what an auditor does. How can you evaluate an auditor’s ability to interview people, take notes, follow audit trails, and manage their time? The most common solution is to require the auditor “shadow” a more experienced auditor several times, and then the trainer will “shadow” the trainee.
Auditor shadowing in 1st party audits
ISO 19011:2018 defines first-party audits as internal audits. When first-party auditors are being shadowed by a trainer or vice versa, there are many opportunities for training. The key to the successful training of auditors is to recognize teachable moments.
When the trainer is auditing, the trainer should look for opportunities to ask the trainee, “What should I do now?” or “What information do I need to record?” In these situations, the trainer asks the trainee what they should do BEFORE doing it. If the trainee is unsure, the trainer should immediately explain what, why, and how with real examples.
When the trainer is shadowing, the trainer should watch and wait for a missed opportunity to gather important information. In these situations, the trainer must resist guiding the trainee until after the trainee appears to be done. When it happens, sometimes the best tool is simply asking, “Are you sure you got all the information you came for?”
Here are five (5) mistakes that I observed trainers made when they were shadowing:
1. Splitting up, instead of staying together, is one of the more common mistakes I have observed. This happens when people are more interested in completing an audit rather than taking advantage of training opportunities. The trainee may be capable of auditing independently, but this is unfair to the trainee because they need feedback on their auditing technique. This is also unfair to the auditee because it is challenging to support multiple auditors simultaneously. When it is unplanned, trainers may not be available for both auditors. If an audit is running behind schedule, this is the perfect time to teach a trainee how to recover sometime in their schedule. Time management is, after all, one of the most challenging skills for auditors to master.
2. Staying in the conference room instead of going to where the work is done is a common criticism of auditors. If the information you need to audit can be found in a conference room, you could have completed the audit remotely. This type of audit only teaches new auditors how to take notes. These are necessary skills that auditors should master in a classroom before shadowing.
3. Choosing an administrative process is a mistake because administrative processes limit the number of aspects of the process approach that an auditor-in-training can practice. Administrative processes rarely have equipment that requires validation or calibration, and the process inputs and outputs consist only of paperwork, forms, or computer records. With raw materials and finished goods to process, the auditor’s job is more challenging because there is more to be aware of.
4. Not providing honest feedback is a huge mistake. Auditors need to be thick-skinned, or they don’t belong in a role where they will criticize others. Before you begin telling others how to improve, you must self-reflect and identify your strengths and weaknesses. Understanding your perspective, strengths, weaknesses, and prejudices is critical to being a practical assessor. As a trainer, it is your job to help new auditors to self-reflect and accurately rate their performance against objective standards.
5. “Silent Shadowing” has no value at all. By this, I mean shadowing another auditor without asking questions. You should mentally pretend you are doing the audit if you are a trainee. Whenever the trainer does something different from how you would do things, you should make a note to ask, “Why did you do that?” If you are the trainer, you should also mentally pretend you are doing the audit. It is not enough to be present. Your job is to identify opportunities for the trainee to improve. The better the trainee, the more challenging it becomes to identify areas for improvement. This is why training other auditors have helped me improve my auditing skills.
Auditor shadowing in 2nd party audits
Auditors responsible for supplier auditing are critical to supplier selection, supplier evaluation, re-evaluation, and the investigation of the root cause for any non-conformities related to a supplier. Auditor shadowing is a great tool to teach supplier auditors and other people responsible for supply-chain management what to look at and what to look for when they audit a supplier. If you are developing a new supplier quality engineer responsible for performing supplier audits, observing the auditor during some actual supplier audits is recommended. Supplier audits are defined as second-party audits in the ISO 19011 Standard. The purpose of these audits is not to verify conformity to all the aspects of ISO 13485. Instead, the primary purpose of these audits is to verify that the supplier has adequate controls to manufacture conforming products for your company consistently. Therefore, processes such as Management Review (Clause 5.6) and Internal Auditing (Clause 8.2.2) are not typically sampled during a second-party audit.
The two most valuable processes for a second-party auditor to sample are 1) incoming inspection and 2) production controls. Using the process approach to auditing, the second-party auditor will have an opportunity to verify that the supplier has adequate controls for documents and records for both of these processes. Training records for personnel performing these activities can be sampled. The adequacy of raw material storage can be evaluated by following the flow of accepted raw materials, leaving the incoming inspection area. Calibration records can be sampled by gathering equipment numbers from calibrated equipment used by both processes. Even process validation procedures can be assessed by comparing the actual process parameters being used in manufacturing with the documented process parameters in the most recent validation or re-validation reports.
I recommend having the trainee shadow the trainer during the process audit of the incoming inspection process and for the trainer to shadow the trainee during the process audit of production processes. The trainee should ask questions between the two process audits to help them fully understand the process approach to auditing. Supplier auditors should also be coached on techniques for overcoming resistance to observing processes involving trade secrets or where competitor products may also be present. During the audit of production processes, the trainer may periodically prompt the trainee to gather the information that will be needed for following audit trails to calibration records, document control, or for comparison with the validated process parameters. The “teachable moment” is immediately after the trainee misses an opportunity, but while the trainee is still close enough to go back and capture the missing details.
Are you allowed to shadow a 3rd party auditor or FDA inspector?
Consider using 3rd party audits and inspections as an opportunity to shadow experienced auditors to learn what they are looking at and what they look for. In addition to shadowing an expert within your own company or an auditor/consultant you hire for an internal audit, you can also shadow a 3rd party auditor or an FDA inspector. This concept was the subject of a discussion thread I ran across on Elsmar Cove from 2005. The comments in the discussion thread supported the idea of shadowing a 3rd party auditor. The process owner (i.e., the manager responsible for that process) should be the guide for whichever process is being audited, and the process owner is responsible for addressing any non-conformities found in the area., The process owner should be present during interviews, but the process owner should refrain from commenting. The 3rd party auditor and the process owner need to know if the person being interviewed was effectively trained and if they can explain the process under the pressure of an audit or FDA inspection. If you are interested in implementing this idea, I recommend using one of two strategies (or both):
- Consider having the internal auditor that audited each process shadow the certification body auditor for the processes they audited during their internal audit. This approach will teach your internal auditor what they might have missed, and they will learn what the 3rd party auditors look for to simulate a 3rd party audit more effectively when conducting internal audits.
- Consider having the internal auditor that is assigned to conduct the next process audit of each process shadow the certification body auditor for that process. This approach will ensure that any nonconformities observed during the 3rd party audit are checked for the effectiveness of corrective actions during the next internal auditor. Your internal auditor will know precisely how the original nonconformity was identified and the context of the finding.
What is a CAPA? How do you evaluate the need to open a new CAPA, and who should be assigned to work on it when you do?
What is a CAPA?
“CAPA” is the acronym for corrective action and preventive action. It’s a systematic process for identifying the root cause of quality problems and identifying actions for containment, correction, and corrective action. In the special case of preventive actions, the actions taken prevent quality problems from ever happening, while the corrective actions prevent quality problems from happening again. The US FDA requires a CAPA procedure, and an inadequate CAPA process is the most common reason for FDA 483 inspection observations and warning letters. When I teach courses on the CAPA process, 100% of the people can tell me what the acronym CAPA stands for. If everyone understands what a CAPA is, why is the CAPA process the most common source of FDA 483 inspection observations and auditor nonconformities?
Most of the 483 inspection observations identify one of the following seven problems:
- the procedure is inadequate
- records are incomplete
- actions planned did not include corrections
- actions planned did not include corrective actions
- actions planned were not taken or delayed
- training is inadequate
- actions taken were not effective
CAPA Resources – Procedures, Forms, and Training
Medical device companies are required to have a CAPA procedure. Medical Device Academy offers a CAPA procedure for sale as an individual procedure or as part of our turnkey quality systems. Purchase of the procedure includes a form for your CAPA records and a CAPA log for monitoring and measuring the CAPA process effectiveness. You can also purchase our risk-based CAPA webinar, which the turnkey quality system includes.
What’s special about preventive action?
I completed hundreds of audits of CAPA processes over the years. Surprisingly, this seems to be a process with more variation from company to company than almost any other process I review. This also seems to be a significant source of non-conformities. In the ISO 13485 Standard, clauses 8.5.2 (Corrective Action) and 8.5.3 (Preventive Action) have almost identical requirements. Third-party auditors, however, emphasize that these are two separate clauses. I like to refer to certification body auditors as purists. Although certification body auditors acknowledge that companies may implement preventive actions as an extension of corrective action, they also expect to see examples of strictly preventive actions.
You may be confused between corrective actions and preventive actions, but there is an easy way to avoid confusion. Ask yourself one question: “Why did you initiate the CAPA?” If the reason was: 1) a complaint, 2) audit non-conformity, or 3) rejected components—then your actions are corrective. You can always extend your actions to include other products, equipment, or suppliers that were not involved if they triggered the CAPA. However, for a CAPA to be purely preventive in nature, you need to initiate the CAPA before complaints, non-conformities and rejects occur.
How do you evaluate the need to open a CAPA?
If the estimated risk is low and the probability of occurrence is known, then alert limits and action limits can be statistically derived. These quality issues are candidates for continued trend analysis—although the alert or action limits may be modified in response to an investigation. If the trend analysis results in identifying events that require action, then that is the time when a formal CAPA should be opened. No formal CAPA is needed if the trend remains below your alert limit.
If the estimated risk is moderate or the probability of occurrence is unknown, then a formal CAPA should be considered. Ideally, you can establish a baseline for the occurrence and demonstrate that frequency decreases upon implementing corrective actions. If you can demonstrate a significant drop in frequency, this verifies the effectiveness of actions taken. If you need statistics to show a difference, then your actions are not effective.
A quality improvement plan may be more appropriate if the estimated risk is high or multiple causes require multiple corrective actions. Two clauses in the Standard apply. Clause 5.4.2 addresses the planning of changes to the Quality Management System. For example, if you correct problems with your incoming inspection process—this addresses 5.4.2. Clause 7.1 addresses the planning of product realization. For example, if you correct problems with a component specification where the incoming inspection process is not effective, this addresses 7.1. The plan could be longer or shorter Depending on the number of contributing causes and the complexity of implementing solutions. If implementing corrective action takes more than 90 days, you might consider the following approach.
Step 1 – open a CAPA
Step 2 – identify the initiation of a quality plan as one of your corrective actions
Step 3 – close the CAPA when your quality plan is initiated (i.e., – documented and approved)
Step 4 –verify effectiveness by reviewing the progress of the quality plan in management reviews and other meeting forums…you can cross-reference the CAPA with the appropriate management review meeting minutes in your effectiveness section
If the corrective action required is installing and validating new equipment, the CAPA can be closed as soon as a validation plan is created. The effectiveness of the CAPA is verified when the validation protocol is successfully implemented, and a positive conclusion is reached. The same approach also works for implementing software solutions to manage processes better. The basic strategy is to start long-term improvement projects with the CAPA system but monitor the status of these projects outside the CAPA system.
Best practices would be implementing six-sigma projects with formal charters for each long-term improvement project.
NOTE: I recommend closing CAPAs when actions are implemented and tracking the effectiveness checks for CAPAs as a separate quality system metric. If closure takes over 90 days, the CAPA should probably be converted to a Quality Plan. This is NOT intended to be a “workaround” to give companies a way to extend CAPAs that are not making progress on time.
Who should be assigned to work on a CAPA?
Personnel in quality assurance are usually assigned to CAPAs, while managers in other departments are less frequently assigned to CAPAs. This is a mistake. Each process should have a process owner, who should be assigned to the root cause investigation, develop a CAPA plan, and manage the planned actions. If the manager is not adequately trained, someone from the quality assurance department should use this as an opportunity to conduct on-the-job training to help them with the CAPA–not do the work for them. This will increase the number of people in the company with CAPA competency. This will also ensure that the process owner takes a leadership role in revising and updating procedures and training on the processes that need improvement. Finally, the process will teach the process owner the importance of using monitoring and measuring the process to identify when the process is out of control or needs improvement. The best practice is to establish a CAPA Board to monitor the CAPA process, expedite actions when needed, and ensure that adequate resources are made available.
What is a root cause investigation?
If you are investigating the root cause of a complaint, people will sample additional records to estimate the frequency of the quality issue. I describe this as investigating the depth of a problem. The FDA emphasizes the need to review other product lines, or processes, to determine if a similar problem exists. I describe this as investigating the breadth of a problem. Most companies describe actions taken on other product lines and/or processes as “preventive actions.” This is not always accurate. If a problem is found elsewhere, actions taken are corrective. If potential problems are found elsewhere, actions taken are preventive. You could have both types of actions, but most people incorrectly identify corrective actions as preventive actions.
Another common mistake is to characterize corrections as corrective actions.
The most striking difference between companies seems to be the number of CAPAs they initiate. There are many reasons, but the primary reason is the failure to use a risk-based approach to CAPAs. Not every quality issue should result in the initiation of a formal CAPA. The first step is to investigate the root cause of a quality issue. The FDA requires that the root cause investigation is documented, but if you already have an open CAPA for the same root cause…DO NOT OPEN A NEW CAPA!!!
What should you do if you do not have a CAPA open for the root cause you identify?
The image below gives you my basic philosophy.
Most CAPA investigations document the estimated probability of occurrence of a quality issue. This is only half of the necessary risk analysis I describe below. Another aspect of an investigation is documenting the severity of potential harm resulting from the quality issue. If a quality issue affects customer satisfaction, safety, or efficacy, the severity is significant. Risk is the product of severity and probability of occurrence.
How much detail is needed in your CAPAs?
One of the most common reasons for an FDA 483 inspection observation related to CAPAs is the lack of detail. You may be doing all the planned tasks but must document your activity. Investigations will often include a lot of detail identifying how the root cause was identified, but you need an equal level of detail for planned containment, corrections, corrective actions, and effectiveness checks. Who is responsible, when will it be completed, how will it be done, what will the records be, and how will you monitor progress? Make sure you include copies of records in the CAPA file as well because this eliminates the need for inspectors and auditors to request additional records that are related to the CAPA. Ideally, the person reviewing the CAPA file will not need to request any additional records. For example, a copy of the revised process procedure, a copy of training records, and a copy of graphed metrics for the process are frequently missing from a CAPA file, but auditors will request this information to verify all actions were completed and that the CAPA is effective.
What is the difference between corrections and corrective actions?
Every nonconformity identified in the original finding requires correction. By reviewing records, FDA inspectors and auditors will verify that each correction was completed. In addition, several new nonconformities may be identified during the investigation of the root cause. Corrections must be documented for the newly found nonconformities as well. Corrective actions are actions you take to prevent new nonconformities from occurring. Examples of the most common corrective actions include: revising procedures, revising forms, retraining personnel, and creating new process metrics to monitor and measure the effectiveness of a process. Firing someone who did not follow a procedure is not a corrective action. Better employee recruiting, onboarding, and management oversight should prevent employees from making serious mistakes. The goal is to have a near-perfect process that identifies human error rather than a near-perfect employee that has to compensate for weak processes.
Implementing timely corrective actions
Every correction and corrective action in your CAPA plan should include a target completion date, and a specific person should be assigned to each task. Once your plan is approved, you need a mechanism for monitoring the on-time completion of each task. There should be top management or a CAPA board this is responsible for reviewing and expediting CAPAs. If CAPAs are being completed on-schedule, regular meetings are short. If CAPAs are behind schedule, management or the CAPA board needs authority and responsibility to expedite actions and make additional resources available when needed. Identifying lead and lag metrics is essential to manage the CAPA process successfully–and all other quality system processes.
What is an effectiveness check?
Implementation of actions and effectiveness of actions is frequently confused. An action was implemented when the action you planned was completed. Usually, this is documented with the approval of revised documents and training records. The effectiveness of actions is more challenging to demonstrate, and therefore it is critical to identify lead and lag metrics for each process. The lead metrics are metrics that measure the routine activities that are necessary for a process, while the lag metrics measure the results of activities. For example, monitoring the frequency of cleaning in a controlled environment is a lead metric, while monitoring the bioburden and particulates is a lag metric. Therefore, effectiveness checks should be quantitative whenever possible. Your effectiveness is weak if you need to use statistics to show a statistical difference before and after implementing your CAPA plan. If a graph of the process metrics is noticeably improved after implementing your CAPA plan, then the effectiveness is strong.
The new FDA goal is to reduce the De Novo review timeline to 150 days for 70% of De Novo submissions, but how long does it take now?
What is an FDA De Novo submission?
An FDA De Novo submission is an application submitted to the FDA for creating a new device product classification. There are three classifications of devices by the FDA: Class 1, Class 2, and Class 3. Class 1 devices are the lowest-risk devices, and they only require general controls. Class 2 devices are moderate-risk devices that require “Special Controls,” and Class 3 are high-risk devices that require Pre-Market Approval (i.e., PMA). De Novo applications can only be submitted for Class 1 and Class 2 devices, and most of the De Novo submissions require clinical data to demonstrate that the clinical benefits of the new device classification outweigh the risks of the device to patients and users. It’s the need for clinical data that is partly responsible for the longer De Novo review timeline.
What is the De Novo review timeline?
Initially, the FDA required that Class 2 devices must be first submitted as a 510k submission. If the device did not meet the criteria for a 510k, then the company could re-submit a De Novo Classification Request to the FDA. On July 9, 2012, the regulations were revised to allow companies to submit De Novo Classification Requests directly. This makes sense because some devices have novel indications for use, and submission of a 510k would be a complete waste of time in money. For example, the first SARS-COV-2 test had to be submitted as a De Novo by Biofire to obtain permanent approval for the test instead of emergency use authorization (EUA). This change in 2012 dramatically reduced the De Novo review timeline.
On October 4, 2021, the FDA published a final rule for De Novo Classification Requests. This new regulation identified the De Novo review timeline as 120 calendar days. Even though 120 days is 30 days longer than the FDA review clock for a 510k, the actual timeline to review De Novo submissions was much longer.
Every five years, when Congress reauthorizes user fee funding of the FDA, new MDUFA goals are established. The draft MDUFA performance goals (which impact FDA funding) were published recently. The specific performance goal to review De Novo submissions is:
FDA will issue a MDUFA decision within 150 FDA Days for 70% of De Novo requests.
There are two problems with this goal. First, the term “FDA Days” is based on calendar days minus the number of days the submission was placed on hold, and we don’t have any visibility into the number of days submissions are placed on hold. In the past, submissions could be placed on hold multiple times during the Refusal to Accept (RTA) screening process, and the “FDA Days” is reset to zero days each time the company receives an RTA hold letter. In addition, even after the submission is finally accepted, the FDA places the submission on hold when they request additional information (i.e., AI Hold). RTA and AI Hold periods can last up to 180 days, and during the Covid-19 pandemic, companies were allowed to extend this up to 360 days.
The second problem with the MDUFA goal is that we only have visibility into the outcome of De Novo submissions that were granted. More than 60 De Novo submissions are submitted each year, but the number of De Novo Classification Requests granted ranged between 21 and 30 over the past three years. Therefore, the 50%+ of De Novo applications denied could skew the % of submissions that meet the MDUFA goal for the De Novo review timeline.
What is the FDA track record in reviewing a De Novo?
Every CEO I speak with asks the same question: “How long does the FDA review take?” In preparation for a webinar I taught about De Novo Classification Requests in 2019, I researched the latest De Novo review timelines. I expected the review timelines to be close to 150 calendar days because the FDA decision goal was 150 FDA days. The 150-day goal was set in 2018 when Congress approved MDUFA IV. The 2019 data held two surprises:
- only 21 De Novo requests were granted in 2019, and
- the average review timeline was 307 calendar days (i.e., the range was 108 days to 619 days).
FDA days are not the same as calendar days. Only 23.8% of De Novo submissions were reviewed within 150 calendar days. The FDA doesn’t calculate the number of FDA days as calendar days, but there is no way to know how much time each De Novo spent on hold publicly. Upon seeing the announcement of a new decision goal for MDUFA V on October 5, 2022, I decided to revisit my previous analysis.
*Only 9+ months of data for 2022, because data was collected on October 17, 2022.
We can blame the Covid-19 pandemic for the slower De Novo review timeline during the past few years, but you would expect a longer average duration in 2020 if that was the root cause of the FDA’s failure to achieve the MDUFA IV target of 150 calendar days. You would also expect 2021 to have the longest review timelines. Instead, the review timelines are the slowest for 2022. The number of De Novo submissions remains small, and therefore it is hard to be conclusive regarding the root cause of the failure to reach the 150-day decision goal. In addition, the percentage of De Novo applications granted within 150 calendar days was lowest in 2021, as you would expect if the reason for delays is primarily due to the Covid-19 pandemic.
Is there any good news?
The FDA is allowing the new eSTAR templates to be used for De Novo Classification Requests. These new electronic submission templates standardize the format of all 510k and De Novo submissions for FDA reviewers. The eSTAR also forces companies to answer all questions in the FDA reviewer’s checklist to ensure the submission is complete and accurate before the new submission is submitted to the FDA.
The new eSTAR templates were first used in 2021, and our firm has observed shorter overall review timelines and fewer deficiencies identified by FDA reviewers when they submit an “Additional Information Hold” (AI Hold) to companies.
How can the FDA improve De Novo timelines?
The FDA, industry, and Congress seem to be taking the same approach pursued five years ago to improve the review timeline for De Novo submission. MDUFA V authorized additional user fees for De Novo submissions (i.e., 17.8% increase), and the FDA will be authorized to hire additional employees each year during MDUFA V if the performance goals are met. However, there are three other options that the FDA and industry should have seriously considered during the FDA-industry negotiations.
The first option that should have been considered is to allow third-party reviewers to review the elements of a De Novo that are identical to a 510k submission:
- sterilization validation
- shelf-life testing
- biocompatibility testing
- software validation
- electrical safety testing
- EMC testing
- wireless testing
- interoperability testing
- benchtop performance testing
- animal performance testing
- human factors engineering
The above approach would require blended pricing where the FDA charges a smaller user fee than a Standard De Novo user fee, and the third-party reviewer charges a smaller fee than a 510k. The combined cost would be higher than the FDA Review of a De Novo, but this would reduce the number of hours the FDA needs to complete their review of a De Novo, and it would allow for pricing that is much lower than the De Novo standard user fee for qualified small businesses.
A second approach would be to pilot a modular review approach. A modular review would be similar to modular reviews for PMA submissions. In a modular review, the FDA can review most submission sections and provide feedback before the human clinical performance data is available. This would not help the few De Novo submissions that do not include human clinical performance data, but this would have a profound positive impact on most De Novo projects. First, the FDA would be able to complete the review of all sections in the submission except the human clinical performance data without delaying the final De Novo decision. Second, a successful review of non-clinical data by the FDA would give investors more confidence to fund pivotal clinical studies required to complete the De Novo submission.
A third approach would be for the FDA to force manufacturers to submit testing plans and protocols as pre-submissions to the FDA. This approach would give the FDA more familiarity with each device and the testing plan before reviewing the data. This approach would also reduce the hours FDA reviewers spend reviewing data that doesn’t meet the requirements and writing deficiencies. This approach would also give investors more confidence to fund De Novo projects for all V&V testing.
A CAPA Board is a team responsible for making sure that all CAPAs are completed on time and the actions taken are effective.
Many of the medical device companies we work with have to open a CAPA for their CAPA process because they fail to implement all the actions that were planned, they fail to implement corrective actions as scheduled, or the actions implemented fail to be effective. When we investigate any process, we typically see one of five common root causes:
- top management is not committed to the CAPA process (we can’t fix this)
- procedures and/or forms are inadequate
- people responsible do not have sufficient training
- management oversight of the process is neglected
- there are not enough resources to do the work
Creating a CAPA Board can address four of these potential root causes, but the CAPA Board needs to understand how to work effectively.
Creating a CAPA Board shows a commitment to quality
Sometimes top management only pays lip service to quality. Top management’s actions demonstrate that quality is a cost-center, and they do not view quality as contributing to the revenue of the company. Instead, quality is viewed as a “necessary evil” like death and taxes. If this describes your company, sharpen your resume and find a new job. Quality is essential to selling medical devices and quality is the responsibility of everyone in the company. The Management Representative is responsible for “ensuring promotion and awareness” (see Clause 5.5.2c of ISO 13485) of regulatory and quality system requirements. This person should be training others on how to implement best practices in quality system management. One person or one department should never be expected to do most of the work related to the quality system.
A CAPA Board should be a cross-functional team of managers that help each other maintain an effective CAPA process. This means: 1) corrections are completed on time, 2) corrective and preventive actions are completed on time, and 3) each CAPA is effective. In order to do this consistently, the CAPA Board needs to work together as a team on the CAPA process. The CAPA Board doesn’t look for someone to blame. Instead, the CAPA Board rotates their responsibilities regularly, everyone is cross-trained on the roles within the CAPA Board, and the team passes tasks from one person or department that is overloaded to another person or department that has the resources to complete the tasks effectively and on time. A professional team must anticipate holes in task coverage, and someone on the team needs to communicate to the rest of the team which hole they are addressing. You can’t wait until the coverage gap is obvious and then have everyone jump into action. If you do this, your effectiveness will resemble a soccer team of 9-year-olds.
Is your CAPA procedure the root cause?
In most companies, the problem is not the CAPA procedure. Clauses 8.5.2 and 8.5.3 of ISO 13485 are quite specific about each step of the CAPA process, and therefore it is easy to write a procedure that includes all of the required elements. The CAPA procedure is also one of the first procedures that auditors and inspectors review, and therefore any deficiencies in your procedure are usually addressed after one or two audits. If you feel that your CAPA procedure needs improvement, the above link explains how to write a better CAPA procedure. You might also consider asking everyone that is responsible for the CAPA process to provide suggestions on how to improve your procedure to streamline the process and clarify the instructions. The best approach is to have a small group (i.e. 3 to 5 people) of middle-level managers, from different departments, assigned to a CAPA Board with the responsibility of improving the CAPA process and procedure. If you have a large company, you might consider rotating people through the CAPA Board each quarter instead of having a larger group.
Does your CAPA Board have sufficient training?
Everyone can benefit from more training–even instructors will periodically engage in refresher training. Before someone is assigned to work on a CAPA, that person needs to be trained. Nobody should be assigned to a CAPA Board unless they are prepared to become an expert in the CAPA process. Some companies will only require people to sign a training record that states they read and understood the CAPA procedure. However, you must also demonstrate that your training was effective and the person is competent at the task assigned. Therefore, we recommend training people on CAPAs by training them with a CAPA training webinar and evaluating the effectiveness of the training by having each person complete a quiz. The use of a training webinar will ensure that each employee receives the same training, and the quiz will provide objective evidence that they understood the training (i.e. it was effective). If you have a CAPA Board, each person on the board should be involved in your CAPA training, and it is their responsibility to make sure people in their department have been trained effectively.
Competency is the hardest thing to demonstrate for any task. You can do this by verifying that the person has performed this task in one or more prior jobs (e.g. resume). If the person does not have evidence of working on CAPAs in their previous employment, then you will need someone that is already competent in the CAPA process to observe each person completing CAPAs and providing feedback. Once each person has demonstrated successful completion of multiple CAPAs, then the expert can attest to their competency in a training record with references to each of the successful CAPAs that were completed. If you are the person assigning a CAPA or individual tasks to people, do not assign the role of investigation, or writing the CAPA, to anyone that has not already demonstrated competency unless you are assessing them for competency. Everyone on the CAPA Board should either already be competent in the CAPA process or another expert on the CAPA Board should be in the process of training them to become a CAPA expert.
CAPA Boards are responsible for management oversight of the CAPA process
The most common method for management oversight of the CAPA process is to discuss the status of CAPAs at a Management Review. This information can be presented by the Management Representative, but assigning the presentation of CAPA status to another person on your CAPA Board will delegate some of the Management Review tasks and gives other people practice at presenting to a group. Some companies only conduct a Management Review once per year, but this makes it impossible to review CAPAs that were initiated immediately after a Management Review unless the CAPA takes more than a year to implement. Even if your company conducts quarterly Management Reviews, the review of CAPA status during a Management Review should focus on the most important issues rather than discuss every CAPA in detail. The impact on safety, the impact on product performance, and the economic impact of a specific CAPA are all criteria for deciding which CAPAs to discuss during a Management Review.
The CAPA Board needs a metric or metrics for monitoring the effectiveness of the CAPA process. The simplest metric is to monitor the average aging of CAPAs. If that average is steadily rising week after week, then new CAPAs are not being initiated, and existing CAPAs are not being closed. You can also measure the time to write a CAPA plan and the time to perform an investigation or monitor the on-time completion of tasks. The most important thing is for someone to take action when these metrics are not aligned with your quality objectives for the CAPA process. Taking action after 90 days of neglect is not good enough. You need to be monitoring the CAPA process weekly, and you need to take action proactively. Therefore, your CAPA Board needs to meet weekly and you need to show evidence in your CAPA records of what actions were taken by the CAPA Board.
Who should be assigned to the CAPA Board?
Top management does not need to be directly involved in the CAPA Board. Top management already reviews the status of CAPAs during Management Reviews. In a small company (i.e. < 20 people) you might have no choice but to have the same people that are assigned to your CAPA Board also be members of top management. As your company gets larger, you should assign middle-level managers and people that are new to management as members of the CAPA Board. Participating in the CAPA Board will teach those managers to work together as a team to achieve shared company goals and to persuade their peers to help them. The experience of working on a CAPA Board will also expose less experienced managers to other departments outside of their expertise. Ideally, participation in the CAPA Board will build friendships between peers that might not speak to one another. Each CAPA represents a team-building opportunity. The team needs to find a way to pool its resources to complete CAPAs on time and effectively. It is also important to rotate the assignment to the CAPA Board so that eventually all of your middle-level managers are trained in the CAPA process and each of them has been evaluated on their demonstration of team leadership and effectiveness in working with peers cooperatively. In large companies, it is common to assign one member of top management to the CAPA Board to show that top management is supportive of the CAPA process and to provide authorization for additional resources and funding for actions when needed. The top management representative should also be rotated to make sure that all of the top management remains competent in the CAPA process.
How does the CAPA Board manage the CAPA process?
The CAPA Board should never be blaming an individual or department for the lack of CAPA success. The CAPA Board should be anticipating when a CAPA is falling behind schedule or might not be as effective as it should be. Nobody on the team should be afraid to voice their opinion or to make a suggestion. Each member of the team has the responsibility of asking for help when they need it and asking for help as early as possible. The CAPA assignments should be shared between the team members, and one person should be responsible for chairing the meetings. If everyone is experienced in participating in CAPA Boards, then the role of the chairperson can be rotated each week. If one or more team members are inexperienced, the person on the CAPA Board assigned to training them should be teaching them how to participate in the meetings and prepare them for acting as chairperson.
Every CAPA Board meeting should have a planned agenda and meeting minutes. Every open CAPA should be discussed during the meeting, but the amount of time devoted to each CAPA should be adjusted for the risk of the CAPA failing to be completed on time or failing to be effective. If a CAPA is going smoothly, the discussion might only last seconds. Any discussion or actions planned that are specific to a CAPA should be documented in the individual CAPA record as well as the meeting minutes. This will ensure that the CAPA records are maintained as required by the ISO 13485 standard and the regulations.