risk analysis Archives

Posts Tagged risk analysis

What are the four types of risk analysis?

Posted by Rob Packard on March 23, 2022

You are familiar with design and process risk analysis, but do you know all four types of risk analysis?

Last week’s YouTube live streaming video answered the question, “What are the four different types of risk analysis?” Everyone in the medical device industry is familiar with ISO 14971:2019 as the standard for medical device risk management, but most of us are only familiar with two or three ways to analyze risks. Most people immediately think that this is going to be a tutorial about four different tools for risk management (e.g. FMEA, Fault Tree Analysis, HAZOP, HACCP, etc.). Instead, this article is describing the four different quality system processes that need risk analysis.

What are the four different types?

The one most people are familiar with is risk analysis associated with the design of a medical device. Do you know what the other three are? The second type is process risk management where you document your risk estimation in a process risk analysis. The third type is part of the medical device software development process, specifically a software hazard analysis. Finally, the fourth type is a Use-Related Risk Analysis (URRA) which is part of your usability engineering and human factors testing. Each type of risk analysis requires different information and there are reasons why you should not combine these into one risk management document or template.

Design Risk Analysis

Design risk analysis is the first type of risk analysis we are reviewing in this article. The most common types of design risk analysis are the design failure modes and effects analysis (dFMEA) and the fault-tree analysis (FTA). The dFMEA is referred to as a bottom-up method because you being by identifying all of the possible failure modes for each component of the medical device and you work your way backward to the resulting effects of each failure mode. In contrast, the FTA is a top-down approach, because you begin with the resulting failure and work your way down to each of the potential causes of the failure. The dFMEA is typically preferred by engineers on a development team because they designed each of the components. However, during a complaint investigation, the FTA is preferred, because you will be informed of the alleged failure of the device by the complainant, but you need to investigate the complaint to determine the cause of the failure. Regardless of which risk analysis tool is used for estimating design risks, the risk management process requires that production and post-production risks be monitored. Therefore, the dFMEA or the FTA will need to be reviewed and updated as post-market data is gathered. If a change to the risk analysis is required, it may also be necessary to update the instructions for use to include new warnings or precautions to prevent use errors.

Process Risk Analysis

Process risk analysis is the second type of risk analysis. The purpose of process risk analysis is to minimize the risk of devices being manufactured incorrectly. The most common method of analyzing risks is to use a process failure modes and effects analysis (i.e. pFMEA). This method is referred to as a bottom-up method because you begin by identifying all of the possible failure modes for each manufacturing process step. Next, the effects of the process failure are identified. After you identify the effects of failure for each process step, the severity of harm is estimated. Then the probability of occurrence of harm is estimated, and the ability to detect the failure is estimated. Each of the three estimates (i.e. Severity, Occurrence, and Detectability) are multiplied to calculate a risk priority number (RPN). The resulting RPN is used to prioritize the development of risk controls for each process step.

As risk controls are implemented, the occurrence and detectability scores estimated again. This is usually where people end the pFMEA process, but to complete one cycle of the pFMEA the risk management team should document the verification of the effectiveness of the risk controls implemented. For example, if the step of the process is sterilization then documentation of effectiveness consists of a sterilization validation report. This is the last step of one cycle in the pFMEA, but the risk management process includes monitoring production and post-production risks. Therefore, as new process failures occur the pFMEA is reviewed to determine if any adjustments are needed in the estimates for severity, occurrence, or detectability. If any of the risks increase, then additional risk controls may be necessary. This process is continuously updated with production and post-production information to ensure that process risks remain acceptable.

Software Hazard Analysis

Sofware hazard analysis is becoming more important to medical devices as physical devices are integrated with hospital information systems and with the development of software as a medical device (SaMD). Software risk analysis is typically referred to as hazard analysis because it is unnecessary to estimate the probability of occurrence of harm. Instead, it is only necessary to identify hazards and estimate harm. Examples of these hazards include loss of communication, mix-up of data, loss of data, etc. For guidance on software hazard identification, IEC/TR 80002-1:2009 is a resource. FDA software validation guidance indicates that software failures are systemic in nature and the probability of occurrence cannot be determined using traditional statistical methods. Therefore, the FDA recommends that you assume that the failure will occur and estimate software risks based on the severity of the hazard resulting from the failure.

Use-Related Risk Analysis

The fourth and final type of risk analysis is use-related risk analysis (URRA). Most development teams assume that they are able to use traditional hazard identification techniques to identify the potential use-related risks. However, use-related risks are inextricably linked to the experiences of the user. The development team has unique knowledge of the device they are developing, and therefore it is likely that use-related risks associated with a lack of knowledge about the device will result in use errors that the development team would not realize. For this reason, formative testing is necessary to identify unforeseen use-related risks. Once formative testing identifies these risks, additional formative usability testing can be used to create and refine the instructions for the use of a medical device. Finally, formative testing can be used to develop user training programs that prevent potential use errors. Once the development team has completed the necessary formative testing, then summative usability testing is used to validate the effectiveness of the risk controls that were implemented.

In the past, I believed that the FDA’s focus on usability was the review of summative usability testing. However, I have learned that the FDA feels it is equally important to begin the human factors testing process by first performing a use-related risk analysis and then identifying the critical tasks. Without identifying these critical tasks, it is not possible for the FDA to determine if the moderator of the summative testing has observed all of the critical tasks being performed correctly. An example of a Use-Related Risk Analysis (URRA) was provided by the FDA in a 510(k) AI deficiency letter that we received. The example is provided below.

URRA table example from the FDA 300x117 What are the four types of risk analysis?

Example of a URRA Table provided by the FDA

Can you use only the IFU to prevent use-related risks?

Instructions for use (IFU) are required to include warnings and precautions. This information provided by the manufacturer explains how to use a medical device correctly and identifies the residual risks. This is a form of risk control, but it is the least effective form of risk control and should be the risk control of last resort. Not everyone reads the IFU, and you cannot guarantee that everyone will understand the instructions. You certainly can’t be sure that users will remember all your warnings or precautions when they are tired, stressed, or acting in an emergency situation. Design controls and protective measures should be implemented as the first and second priority for risk controls, and the IFU should be your lowest priority.

This is the reason why we have color-coding, design features that eliminate the possibility of a use error, we provide training to users, and we are required to monitor use-related risks for medical devices. Formative usability testing is intended to identify use errors we did not anticipate, to help us develop instructions for use (IFU), and help us develop training for users. Summative testing is intended to validate that the design, training, and IFU are effective at preventing use errors. All three of these aspects work together–not the IFU alone. In fact, there is an entire alarms standard that identifies protective measures that shall be used for electromedical devices to prevent use errors (i.e. – IEC 60601-1-8).

Facilitating Risk Management Activities – An Interview with Rick Stockton

I listened to our YouTube video about the four different types of risk analysis, you may have heard my reference to Rick Stockton’s interview that we posted on our YouTube channel and embedded above. In our interview with Rick Stockton, we discussed how to facilitate risk management activities during the design and development of medical devices. If you are interested in learning more about Rick and facilitating risk management activities, please watch the video of our interview with Rick.

Tags: design risk analysis, dFMEA, IEC 62366-1, ISO 14971, ISO/TR 24971, ISO/TR 80002-1, Medical Device, process risk analysis, risk analysis, risk management file, software hazard analysis, URRA, use-related risk analysis

Posted in: ISO 14971:2019 (Risk Management)

Contract Manufacturers Need Strong Risk Management Processes

Posted by Rob Packard on January 5, 2021

This blog discusses why contract manufacturers need to have a strong risk management process, and your company needs to help your contract manufacturers. This article was updated on April 28, 2022, and the original publication was January 05, 2011. Please ignore the date of publication at the top of the post for articles that are more than a year old.

Risk management is not our responsibility Contract Manufacturers Need Strong Risk Management Processes

Can contract manufacturers exclude risk management from the scope of their quality system?

Most contract manufacturers in the medical device industry exclude design from their Quality Management Systems. Unfortunately, most of the contract manufacturers also associate risk management with only the design process. Risk Management cannot be “not applicable” in an ISO 13485 Quality Management System. The requirement of section 7.1 is: “The organization shall establish documented requirements for risk management throughout product realization. Records arising from risk management shall be maintained.” The Standard also references ISO 14971 as a source of guidance on Risk Management.

Medical Device Academy also offers a Two-Part Risk Management Training Webinar for ISO 14971:2019.

Have you experienced an audit dialogue at a contract manufacturer similar to this?

The auditor asks, “How do you manage risk throughout the production process?” Then the auditee responds, “That is the responsibility of our customers. We will prepare a risk analysis if customers pay for it, but usually, customers do the risk analysis.”

For a contract manufacturer, compliance with ISO 14971 is not my primary concern as an auditor. My primary concern is to verify that contract manufacturers analyze risks associated with the processes that they perform and do their best to minimize those risks. What I don’t understand is why more companies don’t want to have strong risk management processes. Risk management is how we prevent bad things from happening. Bad stuff like scrap, complaints, and recalls. Should we expect our suppliers to have a strong risk management process?

Duh.

Why your company needs to be involved in the risk management process?

Risk is a process 1 1024x1024 Contract Manufacturers Need Strong Risk Management Processes

Contract manufacturers should be doing everything they can to get better at risk management. During pre-production planning, they should be asking, “What happens if…” The contract manufacturer knows best HOW things will fail in production, while the customer knows best WHAT happens when things fail in production. To be safe and effective, both companies need to collaborate on risk analysis.

In any risk analysis, you need to estimate the severity of potential harm and the probability of occurrence of that harm. For production defects, the contract manufacturer can estimate the probability of occurrence of defects (i.e., P1 in Annex E of ISO 14971:2007), but the likelihood of occurrence of harm is less. The probability of occurrence of harm is the product of multiplying P1 and P2. The probability that occurrence will result in harm is P2, and P2 is a number that is less than 100% or 1. Your company can gather pre-market clinical data and post-market clinical data to estimate P2, but before launching your product, you can only guess at the value of P2. Your contract manufacturer, however, is not able to estimate P2 at all. It’s ok to estimate risk without P2 during the design phase because this will overestimate risks and result in more conservative decisions.

In addition to P2, your contract manufacturer is also not capable of estimating the severity of potential harm. As the designer of the medical device, you will know best how your device is used and what the likely clinical outcomes are when a device malfunctions. There may even be multiple possible clinical outcomes. The contract manufacturer knows what can go wrong during manufacturing, but you will need to define the clinical outcomes due to malfunctions.

Why do contract manufacturers avoid doing risk analysis?

The reason contract manufacturers avoid doing risk analysis is because it’s time-consuming and tedious.

Too bad, so sad.

Balancing my checkbook is time-consuming and tedious too, but I balance my checkbook to prevent an overdraft charge. Not doing a risk analysis can be much more painful. Scrapping out a part can cost tens or hundreds of dollars. Complaints can cost thousands of dollars. Recalls can cost millions of dollars.

If I owned a contract manufacturing company, I would ensure that everyone in the company is involved in risk management. We don’t want scrap, we can’t afford mistakes that lead to complaints, and a recall could put us out of business.

How Medical Device Academy Can Help?

Medical device academy can help both the contract manufacturer and the specification developer utilizing a contract manufacturer as a supplier! We offer training on 14971:2019 as well as procedures on risk management and supplier quality management.

Two-part Risk Management Training Webinar for ISO 14971:2019 – Part 1 of this webinar will be presented live on Tuesday, March 29 @ 9-10:30 am EDT. Part 2 of this webinar series will be presented live on Tuesday, April 5 at 9-10:30 am EDT. Purchase of this webinar series will grant the customer access to both live webinars. They will also receive the native slide decks and recording for the two webinars.

Tags: contract manufacturers, risk analysis, Risk Management

Posted in: ISO 14971:2019 (Risk Management)

When to Initiate a Corrective And Preventative Action (CAPA)

Posted by Rob Packard on June 15, 2011

This blog reviews the differences between corrective action and preventive action, and when to initiate corrective and preventive action.

I’ve completed almost 100 audits in the past two years, and I review the Corrective Action and Preventive Action (CAPA) process during every single audit. Surprisingly, this seems to be a process with more variation from company to the company than almost any other process I review. This also seems to be a major source of nonconformities. In the ISO 13485 Standard, clause 8.5.2 (Corrective Action) and clause 8.5.3 (Preventive Action) have almost identical requirements. Third-party auditors, however, emphasize that these are two separate clauses. We are purists. Although we acknowledge that companies may implement preventive actions as an extension of corrective action, we also expect to see examples of actions that are strictly preventive in nature.

Many companies seem to be confused, but it doesn’t need to be. Just ask yourself one question. What is the source of this action?

If the answer is a complaint, audit nonconformity, or rejected components—then your actions are corrective.

If the answer is a negative trend that is still within specifications or an “Opportunity For Improvement” (OFI) identified by an auditor—then your actions are preventive.

Root Cause Investigation

If you are investigating the root cause of a complaint, people will sample additional records to estimate the frequency of the quality issue. I describe this as investigating the depth of a problem. The FDA emphasizes the need to review other product lines, or processes, to determine if a similar problem exists. I describe this as investigating the breadth of a problem. Most companies describe actions taken on other product lines and/or processes as “preventive actions.” This is not always accurate. If a problem is found elsewhere, actions taken are corrective. If potential problems are found elsewhere, actions taken are preventive. You could have both types of actions, but most people incorrectly identify corrective actions as preventive actions.

Another common mistake is to characterize corrections as corrective actions.

The most striking difference between companies seems to be the number of CAPAs they initiate. There are many reasons, but the primary reason is the failure to use a risk-based approach to CAPAs. Not every quality issue should result in the initiation of a formal CAPA. The first step is to investigate the root cause of a quality issue. The FDA requires that the root cause investigation is documented, but if you already have an open CAPA for the same root cause…

DO NOT OPEN A NEW CAPA!!!

If you do not have a CAPA open for the root cause that you identify, then what should you do?

I know this will shock everyone, but…it depends.

The image below gives you my basic philosophy.

Most investigations document the estimated probability of occurrence of a quality issue. This is only half of the necessary risk analysis I describe below. Another aspect of an investigation is to document the severity of potential harm resulting from the quality issue. If customer satisfaction, safety, or efficacy are affected by a quality issue—the severity is big. Risk is the product of severity and probability of occurrence.

Estimated Risk-Initiating a Corrective And Preventive Action (CAPA)

If the estimated risk is low and the probability of occurrence is known, then alert limits and action limits can be statistically derived. These quality issues are candidates for continued trend analysis—although the alert limit or action limit may be modified in response to an investigation. If the trend analysis results in identifying events that require action, then that is the time when a formal CAPA should be opened. If the trend remains below your alert limit, then no formal CAPA is needed.

If the estimated risk is moderate or the probability of occurrence is unknown, then a formal CAPA should be considered. Ideally, you will be able to establish a baseline for the occurrence and demonstrate that frequency decreases upon the implementation of corrective actions. If you can demonstrate a significant drop in frequency, this verifies the effectiveness of actions taken. If you need statistics to show a difference, then your actions are not effective.

If the estimated risk is high, or there are multiple causes that require multiple corrective actions, a quality improvement plan may be more appropriate. There are two clauses in the Standard that apply. Clause 5.4.2 addresses the planning of changes to the Quality Management System. For example, if you correct problems with your incoming inspection process—this addresses 5.4.2. Clause 7.1 addresses the planning of product realization. For example, if you correct problems with a component specification where the incoming inspection process is not effective, this addresses 7.1. Depending upon the number of contributing causes and the complexity of implementing solutions, the plan could be longer or shorter. If it will take more than 90 days to implement corrective action, you might consider the following approach.

Step 1 – open a CAPA

Step 2 – identify the initiation of a quality plan as one of your corrective actions

Step 3 – close the CAPA when your quality plan is initiated (i.e., – documented and approved)

Step 4 –verify effectiveness by reviewing the progress of the quality plan in management reviews and other meeting forums…you can cross-reference the CAPA with the appropriate management review meeting minutes in your effectiveness section

If the corrective action required is the installation of new equipment and validating that equipment, the CAPA can be closed as soon as a validation plan is created. The effectiveness of the CAPA is verified when the validation protocol is successfully implemented and a positive conclusion is reached. The same approach also works for implementing software solutions to better manage processes. The basic strategy is to get the long-term improvement projects started with the CAPA system, but monitor the status of these projects outside the CAPA system.

Best practices would be the implementation of six-sigma projects with formal charters for each long-term improvement project.

NOTE: I believe in closing CAPAs when actions are implemented, and tracking the effectiveness checks for CAPAs as a separate quality system metric. If closure takes more than 90 days, the CAPA should probably be converted to a Quality Plan. This is NOT intended to be a “workaround” to give companies a way to extend CAPAs that are not making progress in a timely manner.

Tags: corrective and preventive action (CAPA), risk analysis

Posted in: CAPA