ISO 14971

Benefit-Risk Analysis – ISO 14971:2019, Clause 7.4

8 Comments / ISO 14971:2019 (Risk Management) / By / ,

This article explains the requirements for a benefit-risk analysis as defined in ISO 14971:2019, Clause 7.4 and in the EU regulations.

benefit risk analsyis with scales 1009x1024 Benefit Risk Analysis ISO 14971:2019, Clause 7.4

What is a benefit-risk analysis?

A benefit-risk analysis is one of the risk management activities explained in ISO 14971:2019. Specifically, this requirement is found in clause 7.4 of the medical device risk management standard. Originally, the requirement was described as “risk-benefit analysis” in the second edition of the medical device risk management standard. The US FDA revised their policies for novel devices (e.g., De Novo and PMA submissions) to emphasize that novel devices must demonstrate clinical benefits or they will not be approved. Therefore, the US FDA revised the wording to place the word “benefit” before the word “risk.” This approach and the revised wording was adopted by the committee that was drafting the third edition of the ISO 14971 standard. The wording was also adopted by 2012 European version of the standard, the EU MDR, and EU IVDR. In general, this risk management activity involves a semi-quantitative comparison of clinical benefits with risks of harm. The ISO 14971 standard indicates that if risks are unacceptable, a device can still be recommended for commercial release by a design team if the clinical benefits outweigh the risk of harm.

Is there discretion as to whether a benefit-risk analysis needs to take place?

The ISO 14971 Standard implies that a benefit-risk analysis is only required if the risks of harm exceed a threshold of acceptability. In the ISO/TIR 24971:2020 guidance, the committee clarified that acceptability of risk must be documented in a risk management policy (see Annex C2 for guidance and recommended content for a risk management policy). However, the EU MDR and IVDR regulations require that you perform a benefit-risk analysis for each individual risk and overall residual risk of a medical device. This is stated in Annex I, the General Safety and Performance Requirements. Therefore, if your company distributes devices only in the USA that are Class 1 or Class 2, and the submission type is not a De Novo or Humanitarian Device Exemption (HDE), then you are only required to perform a benefit-risk analysis if the risks of harm are unacceptable. If the device requires a De Novo application, and HDE, or a Class 3 PMA, then you are required to submit a benefit-risk analysis to the FDA in your premarket submission. For companies that distribute devices in Europe, the companies do not have discretion with regard to performing a benefit-risk analysis and they must include it in the risk management file. Since some of Medical Device Academy’s clients are seeking approval for a De Novo, HDE, or PMA, or the companies are distributing in the EU, our risk management procedure does not allow discretion regarding whether a benefit-risk analysis needs to be performed. The template we created for this is TMP-034 in SYS-010. 

As Low As Reasonably Practicable (ALARP)

Your company may have a risk management procedure which includes a matrix for severity and probability. The matrix is probably color-coded to identify red cells as unacceptable risks that require a benefit-risk analysis, yellow cells that are ALARP, and green cells that are acceptable. This practice was criticized in 2012 by the European Commission. “Acceptability” of risk is no longer permitted using the principles of “ALARP.”

Deviation 4 Benefit Risk Analysis ISO 14971:2019, Clause 7.4

The EU regulations require that the analysis of benefit-risk ratios be performed for each risk and all residual risks—not just the risks you identify as unacceptable. However, the EU regulations also do not permit that financial considerations be used as part of the determination of risk acceptability. Financial considerations are implied in the ALARP principle. To clarify this, notes were added to ISO 14971:2019, the guidance on risk acceptability was moved to ISO/TIR 24971:2020, and the concept of ALARP was removed from the risk management standard and the guidance. Therefore, we recommend that your risk management policy reference the need for a benefit-risk analysis, regulatory requirements, the requirements of recognized medical device standards, and stakeholder requirements–not ALARP.

Integrating benefit-risk analysis into your design process

The best way to integrate benefit-risk considerations into your design process is by performing a clinical evaluation. In addition to using clinical literature, clinical study data, and post-market surveillance as inputs for your clinical evaluation, your company should also be using residual risks as inputs to the evaluation. The clinical evaluation should be used to assess the significance of these residual risks, and verify that there are not any risks identified in the clinical evaluation that were not considered in the risk analysis.

In order to document that your company has performed a benefit-risk analysis for each residual risk, you will need to reference the risk management report in the clinical evaluation and vice-versa. Both documents will need to provide traceability to each risk identified in the risk analysis, and conclusions of risk acceptability will need to be based upon the conclusions of the clinical evaluation.

Once your device is commercialized, you will need to update the clinical evaluation with adverse events and other post-market surveillance information. As part of updating clinical evaluations, you will need to determine the acceptability of the risk when weighed against the clinical benefits. These conclusions will then need to be updated in the risk management report—including any new or revised risks.

If you are interested in benefit-risk analysis training, we offer a benefit-risk analysis webinar as part of our 510(k) course series.

Benefit-Risk Analysis – ISO 14971:2019, Clause 7.4 Read More »

How do you demonstrate training competence?

7 Comments / Education, ISO 14971:2019 (Risk Management) / By / ,

Anyone can sign and date training records, but how do you demonstrate the effectiveness of training and competence?
%name How do you demonstrate training competence?

What are the requirements for training?

The requirements for training are found in ISO 13485:2016, Clause 6.2 (i.e., Human Resources). Auditors and inspectors usually only ask for training records, but the requirement for records is the last item in the clause (i.e., “6.2(e) – maintain appropriate records of education, training, skills, and experience (see 4.2.5).”). The first four items in Clause 6.2 all include one word, but it’s not “records.” The most critical word in the requirements is “competence.”

What is the difference between training requirements, training records, training effectiveness, and training competence?

  • Training requirements define the “appropriate education, training, skills, and experience.” Specifically, what degree(s) is required, if any? What training curriculum needs to be completed? What skills are needed to perform the work? And how many years of experience are needed?
  • Training records are documented evidence that all training requirements have been met. Records may include a job application, resume, individual training records, group training records, or a training certificate. Records may also include the results of any quizzes to demonstrate training effectiveness and any evaluation of training competence.
  • Training effectiveness is how well a person understands the information communicated during training. Using terms from human factors engineering (i.e., the PCA Process), the person must have the correct perception and cognition. Cognition can be evaluated by giving people quizzes or asking them questions. Written and verbal exams may also include pictures and/or video. Performing these types of tasks on a quiz or exam are “knowledge tasks.”
  • Training competence is possessing the skill or ability to perform tasks. To demonstrate competence, someone that is already competent (i.e., a subject matter expert) must observe the trainee performing tasks. A person that once possessed the skill or ability can also judge competency, and a person that does not possess the skill or ability can be trained to evaluate skill and ability (e.g., an Olympic Judge or referee). Skills and abilities can be physical or cognitive, but there are also social skills. Examples of each are provided in the table below:

List of abilities and skills 1024x495 How do you demonstrate training competence?

What keeps me awake at night? (a story from April 2, 2011)

I am in Canada, it’s almost midnight, and I can’t sleep. I’m here to teach a Canadian client about ISO 14971—the ISO Standard for Risk Management of medical devices. Most companies requesting this training are doing so for one of two reasons: 1) some design engineers have no risk management training, or 2) the engineers have previous training on risk management, but the training is out-of-date. This Canadian company falls into the second category, and engineers with previous training ask the most challenging questions. This group of engineers forced me to re-read the Standard several times and reflect on the nuances of almost every single phrase. While teaching this risk management course, I learned more about risk management than I ever knew.

The four levels of the Learning Pyramid

The image at the beginning of this blog is a learning model that explains my experiences training Canadian design engineers. I call this model the “Learning Pyramid.” At the base of the pyramid, there are “Newbies.”

This is the first of four levels. At the base, students read policies and procedures, hoping to understand.

The student is now asked to watch someone else demonstrate proper procedures in the second level of the pyramid. One of my former colleagues has a saying that explains the purpose of this process well, “A picture tells a thousand words, but a demonstration is like a thousand pictures.” Our children call this “sharing time,” but everyone over 50 remembers this as “show and tell.”

The student is now asked to perform their tasks in the third level of the pyramid. This is described as “doing,” but in my auditing courses, I refer to this process as “shadowing.” Trainees will first read the procedures for Internal Auditing (level 1). Next, trainees will shadow the trainer during an audit to demonstrate the proper technique (level 2). During subsequent audits, the trainees will audit, and the trainer will shadow the trainee (level 3). During this “doing” phase, the trainer must watch, listen, and wait for what I call the “teachable moment.” This is a moment when the trainee makes a mistake, and you can use this mistake as an opportunity to demonstrate a complex subject.

Finally, in the fourth level of the Learning Pyramid, we now allow the trainee to become a trainer. This is where I am at. I am an instructor, but I am still learning. I am learning what I don’t know.

Teaching forces you back to the bottom of the Learning Pyramid

After teaching, the next step in the learning process is to return to the first level. I re-read the Standards and procedures until I understood the nuances I was unaware of. Then, I search for examples in the real world that demonstrate the complex concepts I am learning. After searching for examples, I tested my knowledge by applying the newly acquired knowledge to an FDA 510(k) or De Novo submission for a medical device client. Finally, I prepared to teach again. This reiterative process reminds me of the game Chutes and Ladders, but one key difference is that we never really reach the level of “Guru.” We continue to improve but never reach our goal of perfection.

Where is training competence in the Learning Pyramid?

Most people feel that a person is competent in a position when they can consistently perform a task. However, the ISO 13485 standard uses the phrase “necessary competence” to suggest that competency for any given task might not be the same for all people. For example, in some cases, it may be sufficient to perform a task with written instructions in front of you, while an instructor might be expected to perform the task without written instructions. The speed at which a person performs a task might also differ. For example, a secretary might be required to touch type at a speed of 60+ wpm with a QWERTY keyboard, while a stenographer must be able to use a STENO keyboard and write at 225 wpm. The accuracy requirements may differ for those two positions as well. Therefore, your company may decide that the training competence requirement for a design engineer is that they can pass an exam on risk management. However, the training competency requirement for the design project lead or Engineering Manager might include teaching inexperienced design engineers to apply the basic risk management principles. Demonstrating the ability to teach inexperienced design engineers might be demonstrated by an auditor interviewing members of your design team.

How do you demonstrate training competence? Read More »

Auditing Risk Management Files

3 Comments / Auditing, CE Marking, ISO 14971:2019 (Risk Management), ISO Auditing, Remote Auditing / By / ,

What do you look at and look for when you are auditing risk management files to ISO 14971 and the new Regulation (EU) 2017/745?

Your cart is empty

Next week, November 15th @ Noon EST, you will have the opportunity to watch a live webinar teaching you what to look at and what to look for when you are auditing risk management files to Regulation (EU) 2017/745 and ISO 14971. Risk Management Files are one of the essential requirements of technical documentation required for CE Marking of medical devices. Most quality system auditors are trained on how to audit to ISO 13485:2016 (or an earlier version of that standard), but very few quality system auditors have the training necessary to audit risk management files.

Why you are not qualified to audit risk management files

Being a qualified lead auditor is not enough to audit the risk management process. When you are auditing a risk management file, you need risk management training and lead auditor training. To audit the risk management process, you will also need training on applicable guidance documents (i.e., ISO/TR 24971:2020) and applicable regulations (i.e., Regulation 2017/745 and/or Regulation 2017/746). There may also be device-specific guidance documents that specify known risks and risk controls that are considered state-of-the-art.

Creating an audit agenda

Once you have scheduled an audit of risk management files, and assigned a lead auditor, then the lead auditor needs to create an audit agenda. The audit can be a desktop audit that is performed remotely, or it can be an on-site audit. Regardless of the approach, the audit should include interviewing participants in the risk management process documented in the risk management file. As a rule of thumb, I expect a minimum of 30 minutes to be spent interviewing the process owner and one or more other participants. Then I spend an additional 60 minutes of auditing time reviewing documents and records.

Your audit agenda should specify the following items at a minimum:

  1. the method of auditing to be used,
  2. date(s) of the audit,
  3. the duration of the audit,
  4. the location of the audit, and
  5. the auditing criteria.

The auditor(s) and the auditee participants should be identified in the audit agenda. Finally, you should specify which documents and records are required for audit preparation. These documents will be used to help identify audit checklist questions and to determine a sampling plan for the audit. At a minimum, you will need a copy of the risk management procedure and a list of the risk management files that are available to audit. You may also want to request the audit plan for each of those risk management files.

What did you look at and look for during your risk management audit?

When you audit the risk management process, you could take any of the following approaches or a combination of more than one. You could audit the process according to the risk management procedure. You could audit the process according to the risk management plan(s) for each risk management file. You could audit using the process approach to auditing. Finally, you could audit in accordance with specific requirements in the ISO 14971:2019 standard and applicable regulations (i.e., Regulation 2017/745). Regardless of which approach you take, your audit notes and the audit report should identify which documents and records you sampled and what you looked for in each document. Providing only a list of the documents is not enough detail.

Creating an auditing checklist for risk management files

Auditors with limited experience are taught to create an audit checklist by creating a table that includes each of the requirements of the audit criteria. For a risk management file, this would include a list of each of the requirements in ISO 14971 for a risk management file (i.e., Clause 9???). However, this approach is more like the approach that you should be using for a gap analysis. The better approach for creating an audit checklist for risk management files is to start by creating a turtle diagram. In the “process inputs” section (i.e., step 2 of 7), you would add questions derived from your review of the risk management plan(s). In the “process outputs” section (i.e., step 3 of 7), you would add questions specific to the risk management report and other records required in a risk management file. In the “with whom” section (i.e., step 5 of 7), you would add questions related to training and competency. You might also identify additional people involved in the risk management process, other than the process owner, to interview as a follow-up trail. In the “how done” section (i.e., step 6 of 7), you would add questions specific to the procedure and forms used for the risk management process. Finally, in the “metrics” section (i.e., 7 of 7), you would verify that the company is conducting risk management reviews and updating risk management documentation in accordance with the risk management procedure and individual risk management plan(s).

Audits are just samples

Just because you can generate a lot of questions for an audit checklist does not mean that you are required to address every question. Audits are intended to be a “spot check” to verify the effectiveness of a process. You should allocate your auditing resources based on the importance of a process and the results of previous audits. I recommend approximately three days for a full quality system audit, and approximately 90-minutes should be devoted to a process unless it is the design control process (i.e., Clause 7.3 of ISO 13485) which typically requires three to four hours due to the importance and complexity of the design controls process. Therefore, you should schedule approximately 30 minutes to interview people for the risk management process and approximately 60 minutes should be reserved for reviewing documents and records. With this limited amount of time, you will not be able to review every record or interview everyone that was involved in the risk management process. This is why auditors always remind auditees that an audit is just a sampling.

Which records are required in a risk management file?

The contents of a risk management file is specified in ISO 14971:2019, Clause 4.5. There are only four bullets in that section, but the preceding sentence says, “In addition to the requirements of other clauses of this document.” Therefore, your risk management file should address all of the requirements in ISO 14971:2019. What I recommend is a virtual risk management folder for each risk management file. As the auditor, you should also request a copy of the risk management policy and procedure. An example of what this would look like is provided below. The numbers in front of each subfolder correspond to the sub-clause or clause for that requirement in ISO 14971:2019.

Risk Management File Example Auditing Risk Management FilesWhich records are most valuable when auditing risk management files?

As an auditor, I typically focus on three types of targets when auditing any process. First, I will sample any corrective actions implemented in response to previous audit findings. Second, will sample documents and records associated with any changes made to the process. Changes would also include any changes that were made to individual risk management files or the creation of a new risk management file. Finally, my third target for audit sampling is any item that I feel is at risk for safety or performance failures. The severity of the safety or performance failure is also considered when prioritizing audit sampling. In the context of a risk management file, I always verify that production and post-production activities are being conducted as planned. I try to verify that risk analysis documentation was reviewed for the need to update the documentation in response to complaints and adverse events.

More auditor training on risk management files

We are recording a live webinar intended to teach internal auditors and consultants how to perform a thorough audit of risk management files against the requirements of the new European Regulation (EU) 2017/745 and ISO 14971.

PXL 20221101 183748328 Auditing Risk Management Files
Auditing Risk Management Files
In this new webinar, you will learn how to conduct a process audit of risk management files. You will learn what to look at and what to look for in order to verify compliance with Regulation (EU) 2017/745 and ISO 14971:2019. The webinar will be approximately one hour in duration. Attendees will be invited to participate in the live webinar and receive a copy of the native slide deck. Anyone purchasing after the live event will receive a link to download the recording of the live event and the native slide deck.
Price: $64.50

In addition to this webinar on auditing risk management files, we also have other risk management training webinars available. The webinar on auditing risk management files will be hosted live on November 15, 2022 @ Noon EST (incorrect in the live video announcement).

Auditing Risk Management Files Read More »

What are the four types of risk analysis?

1 Comment / ISO 14971:2019 (Risk Management) / By / , , , , , , , , , , , ,

You are familiar with design and process risk analysis, but do you know all four types of risk analysis?

Last week’s YouTube live streaming video answered the question, “What are the four different types of risk analysis?” Everyone in the medical device industry is familiar with ISO 14971:2019 as the standard for medical device risk management, but most of us are only familiar with two or three ways to analyze risks. Most people immediately think that this is going to be a tutorial about four different tools for risk management (e.g. FMEA, Fault Tree Analysis, HAZOP, HACCP, etc.). Instead, this article is describing the four different quality system processes that need risk analysis.

What are the four different types?

The one most people are familiar with is risk analysis associated with the design of a medical device. Do you know what the other three are? The second type is process risk management where you document your risk estimation in a process risk analysis. The third type is part of the medical device software development process, specifically a software hazard analysis. Finally, the fourth type is a Use-Related Risk Analysis (URRA) which is part of your usability engineering and human factors testing. Each type of risk analysis requires different information and there are reasons why you should not combine these into one risk management document or template.

Design Risk Analysis

Design risk analysis is the first type of risk analysis we are reviewing in this article. The most common types of design risk analysis are the design failure modes and effects analysis (dFMEA) and the fault-tree analysis (FTA). The dFMEA is referred to as a bottom-up method because you being by identifying all of the possible failure modes for each component of the medical device and you work your way backward to the resulting effects of each failure mode. In contrast, the FTA is a top-down approach, because you begin with the resulting failure and work your way down to each of the potential causes of the failure. The dFMEA is typically preferred by engineers on a development team because they designed each of the components. However, during a complaint investigation, the FTA is preferred, because you will be informed of the alleged failure of the device by the complainant, but you need to investigate the complaint to determine the cause of the failure. Regardless of which risk analysis tool is used for estimating design risks, the risk management process requires that production and post-production risks be monitored. Therefore, the dFMEA or the FTA will need to be reviewed and updated as post-market data is gathered. If a change to the risk analysis is required, it may also be necessary to update the instructions for use to include new warnings or precautions to prevent use errors.

Process Risk Analysis

Process risk analysis is the second type of risk analysis. The purpose of process risk analysis is to minimize the risk of devices being manufactured incorrectly. The most common method of analyzing risks is to use a process failure modes and effects analysis (i.e. pFMEA). This method is referred to as a bottom-up method because you begin by identifying all of the possible failure modes for each manufacturing process step. Next, the effects of the process failure are identified. After you identify the effects of failure for each process step, the severity of harm is estimated. Then the probability of occurrence of harm is estimated, and the ability to detect the failure is estimated. Each of the three estimates (i.e. Severity, Occurrence, and Detectability) are multiplied to calculate a risk priority number (RPN). The resulting RPN is used to prioritize the development of risk controls for each process step.

As risk controls are implemented, the occurrence and detectability scores estimated again. This is usually where people end the pFMEA process, but to complete one cycle of the pFMEA the risk management team should document the verification of the effectiveness of the risk controls implemented. For example, if the step of the process is sterilization then documentation of effectiveness consists of a sterilization validation report. This is the last step of one cycle in the pFMEA, but the risk management process includes monitoring production and post-production risks. Therefore, as new process failures occur the pFMEA is reviewed to determine if any adjustments are needed in the estimates for severity, occurrence, or detectability. If any of the risks increase, then additional risk controls may be necessary. This process is continuously updated with production and post-production information to ensure that process risks remain acceptable.

Software Hazard Analysis

Sofware hazard analysis is becoming more important to medical devices as physical devices are integrated with hospital information systems and with the development of software as a medical device (SaMD). Software risk analysis is typically referred to as hazard analysis because it is unnecessary to estimate the probability of occurrence of harm. Instead, it is only necessary to identify hazards and estimate harm. Examples of these hazards include loss of communication, mix-up of data, loss of data, etc. For guidance on software hazard identification, IEC/TR 80002-1:2009 is a resource. FDA software validation guidance indicates that software failures are systemic in nature and the probability of occurrence cannot be determined using traditional statistical methods. Therefore, the FDA recommends that you assume that the failure will occur and estimate software risks based on the severity of the hazard resulting from the failure. 

Use-Related Risk Analysis

The fourth and final type of risk analysis is use-related risk analysis (URRA). Most development teams assume that they are able to use traditional hazard identification techniques to identify the potential use-related risks. However, use-related risks are inextricably linked to the experiences of the user. The development team has unique knowledge of the device they are developing, and therefore it is likely that use-related risks associated with a lack of knowledge about the device will result in use errors that the development team would not realize. For this reason, formative testing is necessary to identify unforeseen use-related risks. Once formative testing identifies these risks, additional formative usability testing can be used to create and refine the instructions for the use of a medical device. Finally, formative testing can be used to develop user training programs that prevent potential use errors. Once the development team has completed the necessary formative testing, then summative usability testing is used to validate the effectiveness of the risk controls that were implemented.
 
In the past, I believed that the FDA’s focus on usability was the review of summative usability testing. However, I have learned that the FDA feels it is equally important to begin the human factors testing process by first performing a use-related risk analysis and then identifying the critical tasks. Without identifying these critical tasks, it is not possible for the FDA to determine if the moderator of the summative testing has observed all of the critical tasks being performed correctly. An example of a Use-Related Risk Analysis (URRA) was provided by the FDA in a 510(k) AI deficiency letter that we received. The example is provided below.
URRA table example from the FDA 300x117 What are the four types of risk analysis? Example of a URRA Table provided by the FDA
 

Can you use only the IFU to prevent use-related risks?

Instructions for use (IFU) are required to include warnings and precautions. This information provided by the manufacturer explains how to use a medical device correctly and identifies the residual risks. This is a form of risk control, but it is the least effective form of risk control and should be the risk control of last resort. Not everyone reads the IFU, and you cannot guarantee that everyone will understand the instructions. You certainly can’t be sure that users will remember all your warnings or precautions when they are tired, stressed, or acting in an emergency situation. Design controls and protective measures should be implemented as the first and second priority for risk controls, and the IFU should be your lowest priority.
 
This is the reason why we have color-coding, design features that eliminate the possibility of a use error, we provide training to users, and we are required to monitor use-related risks for medical devices. Formative usability testing is intended to identify use errors we did not anticipate, to help us develop instructions for use (IFU), and help us develop training for users. Summative testing is intended to validate that the design, training, and IFU are effective at preventing use errors. All three of these aspects work together–not the IFU alone. In fact, there is an entire alarms standard that identifies protective measures that shall be used for electromedical devices to prevent use errors (i.e. – IEC 60601-1-8).

 

Facilitating Risk Management Activities – An Interview with Rick Stockton

I listened to our YouTube video about the four different types of risk analysis, you may have heard my reference to Rick Stockton’s interview that we posted on our YouTube channel and embedded above. In our interview with Rick Stockton, we discussed how to facilitate risk management activities during the design and development of medical devices. If you are interested in learning more about Rick and facilitating risk management activities, please watch the video of our interview with Rick.
 

What are the four types of risk analysis? Read More »

Risk Control Selection – Deviation #5 in ISO 14971

2 Comments / ISO 14971:2019 (Risk Management) / By / , , , ,

ISO 14971:2012 deviation #5 is specific to selecting risk control options and protective measures for CE Marking medical devices.

%name Risk Control Selection Deviation #5 in ISO 14971If your company is CE Marking medical devices, you are required to satisfy the Essential Requirements for Safety and Performance as defined in the three European Directives: the MDD, the AIMD, and the IVDD. Throughout these Essential Requirements, there is a requirement to reduce risks “as far as possible” (AFAP) by implementing risk controls. At one time, the expectation was for companies to implement state of the art concerning risk controls, and “state of the art” was interpreted as the latest version of the harmonized ISO Standards. However, lawyers dominating the European Commission appear to disagree with the status quo.

Therefore, in 2012, the European National (EN) version of the Medical Device Risk Management Standard was revised. There is no change to the content of Clauses 1 through 9. Instead, the European Commission identified seven content deviations between the ISO 14971 Standard and the EU Directives. These deviations are identified and explained in Annexes ZA, ZB, and ZC. This blog is the fifth installment of Medical Device Academy’s seven-part blog series on this topic. The goal of the series is to identify solutions for meeting the Essential Requirements by suggesting changes to the current best practices of implementing a risk management process for medical device design.

Discretion as to the Risk Control Options/Measures

Essential Requirements 1 and 2 require that risk control options are implemented for all risks before determining the acceptability of residual risks. The 2nd Essential requirement also requires manufacturers to implement all risk control options—unless the risk controls do not further reduce risk.iso14971 deviation 5 Risk Control Selection Deviation #5 in ISO 14971

Clause 6.2 of the 14971 Standard suggests that you only need to use “one or more” of the risk control options, and Clause 6.4 indicates that further risk control measures are not required if the risk is acceptable. There is an apparent contradiction between the intent of the Standard and the Directives.

If risk acceptability has no impact on whether you will implement risk controls, there is no need for performing a preliminary risk evaluation. Therefore, I have three recommendations for changes to your current risk management process:

  1. Ignore Clause 5 of the 2007/2009 version of ISO 14971
  2. Eliminate the second step of risk assessment from your flow chart for risk management (see Figure 1 from the 14971 Standard)
  3. Define risk management policies upon clinical benefits, rather than absolute risks

Instead of performing a preliminary risk evaluation (Clause 6.5), risk/benefit analysis should be moved to Clause 7, where the evaluation of overall residual risk acceptability is required. By making this change, risk controls will be implemented, regardless of risk acceptability, and the acceptability of risks will be dependent upon the risk/benefit analysis alone.

Impact of this Deviation

Implementing changes to your risk management process to address this deviation has great potential to impact the design of devices—not just the risk management documentation. Design teams will no longer be able to stop the design process with an initial design that has an “acceptable risk.” Instead, design teams will be forced to implement additional risk controls and protective measures for device designs that already have a low risk of harm for specific failure modes.

The requirement to implement additional risk controls will increase the cost of devices that may have been relatively safe without the risk controls. For example, if a device is not intended to be implanted, it is a potential foreseeable misuse. Your company may have used the instructions for use to communicate the residual risk associated with misuse of the device. However, now your company will have to implement design controls (e.g., –a selection of materials suitable for implantation) to eliminate the risks associated with misuse and protective measures (e.g., – radio-opaque thread) to help retrieve product that was implanted in an “off-label” usage.

If you are interested in risk management training, Medical Device Academy offers a risk management training webinar.

Risk Control Selection – Deviation #5 in ISO 14971 Read More »

ALARP vs As far as possible – Deviation #3

3 Comments / ISO 14971:2019 (Risk Management) / By / ,

This third blog in a seven-part series reviews deviation #3, ALARP vs. “As far as possible,” with regard to risk reduction.

chart dev 3 ALARP vs As far as possible Deviation #3In 2012, the European National (EN) version of the Medical Device Risk Management Standard was revised, but there was no change to the content of Clauses 1 through 9. Instead, the European Commission identified seven content deviations between the 14971 Standard and the requirements of three device directives for Europe. This seven-part blog series reviews each of these changes individually and recommends changes to be made to your current risk management policies and procedure.

Note: This is 2013 blog that will be updated in the near future, but the following link is for our current risk management training.

Risk reduction: “As Far As Possible” (AFAP) vs. “As Low As Reasonably Practicable” (ALARP)

The third deviation is specific to the reduction of risk. Design solutions cannot always eliminate risk. This is why medical devices use protective measures (i.e., – alarms) and inform users of residual risks (i.e., – warnings and contraindications in an Instructions For Use (IFU). However, Essential Requirement 2 requires that risks be reduced “as far as possible.” Therefore, it is not acceptable to only reduce risks with cost-effective solutions. The “ALARP” concept has a legal interpretation, which implies financial considerations. However, the European Directives will not allow financial considerations to override the Essential Requirements for the safety and performance of medical devices. If risk controls are not implemented, the justification for this must be on another basis other than financial.

There are two acceptable reasons for not implementing certain risk controls. First, risk control will not reduce additional risk. For example, if your device already has one alarm to identify a battery failure, a second alarm for the same failure will not reduce further risk. The redundant alarms are often distracting, and too many alarms will result in users ignoring them.

The second acceptable reason for not implementing a risk control is that there is a more effective risk control that cannot be simultaneously implemented. For example, there are multiple ways to anchor orthopedic implants to bone. However, there is only enough real estate to have one fixation element at each location. If a femoral knee implant is already being anchored to the femur with metal posts and bone cement, you cannot also use bone screws at the same location on the femur to anchor the implants in place.

ALARP does not reduce risk “As far as possible”

Annex D.8 in ISO 14971, recommends the ALARP concept in Clause 3.4 of the 14971 Standard. Therefore, the risk management standard is contradicting the MDD. This contradiction is the primary reason why medical device companies should discontinue the use of phthalates and latex for most medical devices. Even though these materials are inexpensive solutions to many engineering challenges presented by medical devices, these materials present risks that can be avoided by using more expensive materials that are not hazardous and do not pose allergic reactions to a large percentage of the population. The use of safer materials is considered “state-of-the-art,” and these materials should be implemented if the residual risks, after implementation of the risk control (i.e., – use of a safer material) are not equal to, or greater than, the risk of the cheaper material.

Recommendation for eliminating ALARP

Your company may have created a risk management procedure that includes a matrix for severity and probability. The matrix is probably color-coded to identify red cells as unacceptable risks, yellow cells that are ALARP, and green cells that are acceptable. To comply with EN ISO 14971:2012, the “yellow zone” should not be labeled as ALARP. A short-term solution is to simply re-label these as high, medium, and low risks. Unfortunately, renaming the categories of risk high, medium, and low will not provide guidance as to whether the residual risk is reduced “as far as possible.”

Resolution to this deviation

As companies become aware of this deviation between the 14971 Standard and the Essential Requirements of the device directives, teams that are working on risk analysis, and people that are performing a gap analysis of their procedures will need to stop using a matrix, like the example above. Instead of claiming that the residual risks are ALARP, your company will need to demonstrate that risks are reduced AFAP, by showing objective evidence that all possible risk control options were considered and implemented. Your procedure or work instruction for performing a risk control option analysis may currently state that you will apply your risk management policy to determine if additional risk controls need to be applied, or if the residual risks are ALARP.

This procedure or work instruction needs to be revised to specify that all risk control options will be implemented unless the risk controls would not reduce risks further, or the risk controls are incompatible with other risk controls. Risk control options should never be ruled out due to cost.

ALARP vs As far as possible – Deviation #3 Read More »

Risk Acceptability – Deviation #2 in EN ISO 14971

6 Comments / ISO 14971:2019 (Risk Management) / By /

This 7-part blog series continues with the author reviewing deviation #2, risk acceptability, in the EN ISO 14971:2012 Standard.
%name Risk Acceptability Deviation #2 in EN ISO 14971

In 2012, the European National (EN) version of the Medical Device Risk Management Standard was revised, but there was no change to the content of Clauses 1 through 9. Instead, the European Commission identified seven content deviations between the 14971 Standard and the requirements of three device directives for Europe. This seven-part blog series reviews each of these changes individually. The second deviation is specific to risk acceptability.

Discretionary power of manufacturers as to Risk Acceptability

The second deviation is specific to determining risk acceptability in the risk evaluation process. The ISO 14971 Standard indicates in Annex D4 that the acceptability of risk is not specified by the Standard and must be determined by the manufacturer. Clause 3.2 of the 14971 Standard, it states that “Top management  shall: define and document the policy for determining criteria for risk acceptability.” This risk management policy is intended to indicate a threshold for risk acceptability. In Clause 5 of the 14971 Standard, the manufacturer is instructed to evaluate whether risks are acceptable using the risk management criteria defined in the risk management policy.

Essential requirements 1 and 2 require that risks be reduced as far as possible and that all risks shall be included in a risk-benefit analysis—not just the risks that exceed a certain threshold for risk acceptability. Therefore, the requirement to establish a risk policy for the acceptability of risk directly contradicts the MDD.

Since the 2nd edition of the 14971 Standard was first issued (i.e., -2007), clients have been asking me how to establish risk acceptability criteria, for new devices, I recommend benchmarking the risks of the new device against existing devices. In other words, if the new device presents equal or lower risks than existing devices, then the risks of the new device are acceptable. For existing devices, I recommend performing a risk-benefit analysis, evaluating adverse events observed with the device against the benefits of using the device. Unfortunately, most companies choose arbitrary thresholds for risk acceptability. Instead of relying upon benchmarking or risk-benefit analysis, companies will establish a policy that all risks must be below a quantitative value. For example, if the range of possible risk scores is from 1 to 1,000, all risks of 100 or less may be acceptable.

What is acceptable?

In order to comply with the EN ISO 14971:2012 version of the risk management standard, you will need to implement risk controls for all risks, regardless of acceptability. However, you will also need to perform a risk-benefit analysis. The risk-benefit analysis should consider not only the benefits to patients and the risks of using the device, but the analysis should also consider the relative benefits of using other devices.

The clinical evaluation report and the risk management report for the device should be based upon clinical evidence of the device for the intended use—including adverse events. For new devices that are evaluated based upon a literature review of equivalent devices, Notified Bodies expect a Post-Market Clinical Follow-up (PMCF) study to be conducted to verify that the actual risk-benefit of the device is consistent with the conclusions of the clinical evaluation. To perform this analysis, a clinical expert is necessary to properly evaluate the risk-benefit ratio of the device and to create a protocol for a PMCF study.

MEDDEV 2.12/2 rev 2, Post Market Clinical Follow-up Studies, indicates that the PMCF study protocol should indicate the study endpoints and the statistical considerations. In order to do this, your company will need to establish quantitative criteria for the acceptability of the identified risks. Therefore, the current 14971 Standard needs to be modified to clarify that risk acceptability criteria should be based upon clinical data, and evaluation of risks should be conducted at a later point in the risk management process (e.g., – as part of the overall risk-benefit analysis).

Impact of Deviation #2

As your company becomes aware of the second deviation between the 14971 Standard and the Essential Requirements of the device directives, your risk management team will need to change the risk management process to clarify when risk acceptability should be evaluated, and the risk management policy should specify how acceptability should be determined.

The risk management process at your company will need to specify that the implementation of risk controls is required for all risks—regardless of acceptability. You should also consider eliminating the evaluation of risk before the implementation of risk controls. Instead, your company should base the acceptability of risk solely upon the clinical risk-benefit analysis and should involve the manufacturer’s medical officer in making this determination.

Finally, your risk management process should specify the need for PMCF studies in order to verify that actual clinical data supports the conclusion that the risk-benefit ratio is acceptable over the lifetime of the device.

If you are interested in ISO 14971 training, we are conducting a risk management training webinar on October 19, 2018.

Risk Acceptability – Deviation #2 in EN ISO 14971 Read More »

Negligible Risks – Deviation #1 in ISO 14971

4 Comments / ISO 14971:2019 (Risk Management) / By /

This blog reviews the treatment of the negligible risks, which is deviation #1 within the EN ISO 14971:2012 European normative risk management standard.

%name Negligible Risks Deviation #1 in ISO 14971

In 2012, the European National (EN) version of the Medical Device Risk Management Standard was revised, but there was no change to the content of Clauses 1 through 9. Instead, the European Commission identified seven content deviations between the 14971 Standard and the requirements of three device directives for Europe. This seven-part blog series reviews each of these changes individually.

Treatment of Negligible Risks in ISO 14971

The first deviation is specific to the treatment of negligible risks. In Annex D8.2, the ISO 14971 Standard indicates that the manufacturer may discard negligible risks. However, Essential Requirements in the three device directives require that “All risks, regardless of their dimension, need to be reduced as much as possible and need to be balanced, together with all other risks, against the benefit of the device.”

Common Misinterpretations

One of the most common mistakes is to confuse the concepts of hazard, harm, and risk. Each of these terms is defined in the ISO 14971 Standard in section 2, but the common mistake is to think that the European Commission is saying that 100% of the hazards you identify need to be reduced as much as possible.

The intent is to require manufacturers to reduce risks, rather than hazards. The first step of the risk analysis process involves identifying hazards. Still, some of these hazards may never result in harm, due to risk controls that are inherent to the design your company has chosen. Also, the severity of harm that a hazard may present could be so low that it may present no risk to the user or patient.

The best practice in risk management is to identify as many hazards as possible at the beginning of the risk analysis process. Still, then these hazards must be sorted into those hazards that will be analyzed for risk. One of the common phrases used in training is: “It is better to estimate the risk of 10% of 1,000 hazards than it is to estimate 50% of 100 hazards.”

If you follow the logic behind the phrase above, your team will need to estimate risk for 100 hazards, rather than 50 hazards. Your risk analysis team will also need to document the rationale behind the categorization of hazards.

Categorizing Hazards

If a hazard is associated with adverse events in the Manufacturer and User Facility Device Experience (MAUDE) database for your device or a similar device, then you need to ensure that the risk associated with that hazard is assessed and there are adequate risk controls. This is also true for any hazard associated with a customer complaint that your company anticipates. Any hazard that presents a high potential severity of harm should also be included in your risk analysis. However, if a hazard is entirely eliminated by the design of your device, then you do not need to include it in the risk analysis.

I recommend writing a hazard identification report that includes all the hazards that were identified. This report should also categorize the hazard. You only need two categories: 1) hazards to be analyzed for risk, and 2) hazards that do not require risk analysis. You need a rationale for each risk that you do not perform risk analysis for, and you need traceability to risk controls and the risk-benefit analysis for each hazard that you do analyze.

Example of a Rationale for Not Analyzing the Risk of a Hazard

About eight years ago, the United States Food and Drug Administration (USFDA) issued an alert cautioning physicians to avoid the use of hemostatic agents near the spinal column, due to the potential hazard of paralysis caused by the swelling of a hemostatic agent as it absorbs the blood. My employer, Z-Medica, quickly received many customer inquiries asking about the safety of QuikClot near the spinal column. I was able to quickly respond that there were zero risks of QuikClot causing paralysis because that particular hemostatic agent did not swell. Instead of absorption, the product adsorbed blood and did not change in size or shape during the adsorption process.

Impact of Deviation #1 about “Negligible Risk”

As companies become aware of this deviation between the 14971 Standard and the Essential Requirements of the device directives, I believe teams that are working on risk analysis and people that are performing a gap analysis of their procedures will need to be more careful about which hazards are identified in their risk management reports. The burden of showing traceability from hazards to risk controls and risk-benefit analysis is substantial. Therefore, it is important to be systematic about how hazards are identified and to provide a clear justification for any hazards that are not included in the risk analysis.

The common phrase that has been used in risk management training classes should be reconsidered in light of feedback from the European Commission. Maybe a better phrase would be: “It is better to estimate the risk of 10% of 200 hazards than it is to estimate 50% of 20 hazards. However, it is important to provide a clear justification for any hazards that are not included in the risk analysis.”

If you are interested in ISO 14971 training, we are conducting a risk management training webinar on October 19, 2018.

Negligible Risks – Deviation #1 in ISO 14971 Read More »

Scroll to Top