Anyone can sign and date training records, but how do you demonstrate the effectiveness of training and competence?
What are the requirements for training?
The requirements for training are found in ISO 13485:2016, Clause 6.2 (i.e., Human Resources). Auditors and inspectors usually only ask for training records, but the requirement for records is the last item in the clause (i.e., “6.2(e) – maintain appropriate records of education, training, skills, and experience (see 4.2.5).”). The first four items in Clause 6.2 all include one word, but it’s not “records.” The most critical word in the requirements is “competence.”
What is the difference between training requirements, training records, training effectiveness, and training competence?
Training requirements define the “appropriate education, training, skills, and experience.” Specifically, what degree(s) is required, if any? What training curriculum needs to be completed? What skills are needed to perform the work? And how many years of experience are needed?
Training records are documented evidence that all training requirements have been met. Records may include a job application, resume, individual training records, group training records, or a training certificate. Records may also include the results of any quizzes to demonstrate training effectiveness and any evaluation of training competence.
Training effectiveness is how well a person understands the information communicated during training. Using terms from human factors engineering (i.e., the PCA Process), the person must have the correct perception and cognition. Cognition can be evaluated by giving people quizzes or asking them questions. Written and verbal exams may also include pictures and/or video. Performing these types of tasks on a quiz or exam are “knowledge tasks.”
Training competence is possessing the skill or ability to perform tasks. To demonstrate competence, someone that is already competent (i.e., a subject matter expert) must observe the trainee performing tasks. A person that once possessed the skill or ability can also judge competency, and a person that does not possess the skill or ability can be trained to evaluate skill and ability (e.g., an Olympic Judge or referee). Skills and abilities can be physical or cognitive, but there are also social skills. Examples of each are provided in the table below:
What keeps me awake at night? (a story from April 2, 2011)
I am in Canada, it’s almost midnight, and I can’t sleep. I’m here to teach a Canadian client about ISO 14971—the ISO Standard for Risk Management of medical devices. Most companies requesting this training are doing so for one of two reasons: 1) some design engineers have no risk management training, or 2) the engineers have previous training on risk management, but the training is out-of-date. This Canadian company falls into the second category, and engineers with previous training ask the most challenging questions. This group of engineers forced me to re-read the Standard several times and reflect on the nuances of almost every single phrase. While teaching this risk management course, I learned more about risk management than I ever knew.
The four levels of the Learning Pyramid
The image at the beginning of this blog is a learning model that explains my experiences training Canadian design engineers. I call this model the “Learning Pyramid.” At the base of the pyramid, there are “Newbies.”
This is the first of four levels. At the base, students read policies and procedures, hoping to understand.
The student is now asked to watch someone else demonstrate proper procedures in the second level of the pyramid. One of my former colleagues has a saying that explains the purpose of this process well, “A picture tells a thousand words, but a demonstration is like a thousand pictures.” Our children call this “sharing time,” but everyone over 50 remembers this as “show and tell.”
The student is now asked to perform their tasks in the third level of the pyramid. This is described as “doing,” but in my auditing courses, I refer to this process as “shadowing.” Trainees will first read the procedures for Internal Auditing (level 1). Next, trainees will shadow the trainer during an audit to demonstrate the proper technique (level 2). During subsequent audits, the trainees will audit, and the trainer will shadow the trainee (level 3). During this “doing” phase, the trainer must watch, listen, and wait for what I call the “teachable moment.” This is a moment when the trainee makes a mistake, and you can use this mistake as an opportunity to demonstrate a complex subject.
Finally, in the fourth level of the Learning Pyramid, we now allow the trainee to become a trainer. This is where I am at. I am an instructor, but I am still learning. I am learning what I don’t know.
Teaching forces you back to the bottom of the Learning Pyramid
After teaching, the next step in the learning process is to return to the first level. I re-read the Standards and procedures until I understood the nuances I was unaware of. Then, I search for examples in the real world that demonstrate the complex concepts I am learning. After searching for examples, I tested my knowledge by applying the newly acquired knowledge to an FDA 510(k) or De Novo submission for a medical device client. Finally, I prepared to teach again. This reiterative process reminds me of the game Chutes and Ladders, but one key difference is that we never really reach the level of “Guru.” We continue to improve but never reach our goal of perfection.
Where is training competence in the Learning Pyramid?
Most people feel that a person is competent in a position when they can consistently perform a task. However, the ISO 13485 standard uses the phrase “necessary competence” to suggest that competency for any given task might not be the same for all people. For example, in some cases, it may be sufficient to perform a task with written instructions in front of you, while an instructor might be expected to perform the task without written instructions. The speed at which a person performs a task might also differ. For example, a secretary might be required to touch type at a speed of 60+ wpm with a QWERTY keyboard, while a stenographer must be able to use a STENO keyboard and write at 225 wpm. The accuracy requirements may differ for those two positions as well. Therefore, your company may decide that the training competence requirement for a design engineer is that they can pass an exam on risk management. However, the training competency requirement for the design project lead or Engineering Manager might include teaching inexperienced design engineers to apply the basic risk management principles. Demonstrating the ability to teach inexperienced design engineers might be demonstrated by an auditor interviewing members of your design team.
What do you look at and look for when you are auditing risk management files to ISO 14971 and the new Regulation (EU) 2017/745?
Your cart is empty
Next week, November 15th @ Noon EST, you will have the opportunity to watch a live webinar teaching you what to look at and what to look for when you are auditing risk management files to Regulation (EU) 2017/745 and ISO 14971. Risk Management Files are one of the essential requirements of technical documentation required for CE Marking of medical devices. Most quality system auditors are trained on how to audit to ISO 13485:2016 (or an earlier version of that standard), but very few quality system auditors have the training necessary to audit risk management files.
Why you are not qualified to audit risk management files
Being a qualified lead auditor is not enough to audit the risk management process. When you are auditing a risk management file, you need risk management training and lead auditor training. To audit the risk management process, you will also need training on applicable guidance documents (i.e., ISO/TR 24971:2020) and applicable regulations (i.e., Regulation 2017/745 and/or Regulation 2017/746). There may also be device-specific guidance documents that specify known risks and risk controls that are considered state-of-the-art.
Creating an audit agenda
Once you have scheduled an audit of risk management files, and assigned a lead auditor, then the lead auditor needs to create an audit agenda. The audit can be a desktop audit that is performed remotely, or it can be an on-site audit. Regardless of the approach, the audit should include interviewing participants in the risk management process documented in the risk management file. As a rule of thumb, I expect a minimum of 30 minutes to be spent interviewing the process owner and one or more other participants. Then I spend an additional 60 minutes of auditing time reviewing documents and records.
Your audit agenda should specify the following items at a minimum:
the method of auditing to be used,
date(s) of the audit,
the duration of the audit,
the location of the audit, and
the auditing criteria.
The auditor(s) and the auditee participants should be identified in the audit agenda. Finally, you should specify which documents and records are required for audit preparation. These documents will be used to help identify audit checklist questions and to determine a sampling plan for the audit. At a minimum, you will need a copy of the risk management procedure and a list of the risk management files that are available to audit. You may also want to request the audit plan for each of those risk management files.
What did you look at and look for during your risk management audit?
When you audit the risk management process, you could take any of the following approaches or a combination of more than one. You could audit the process according to the risk management procedure. You could audit the process according to the risk management plan(s) for each risk management file. You could audit using the process approach to auditing. Finally, you could audit in accordance with specific requirements in the ISO 14971:2019 standard and applicable regulations (i.e., Regulation 2017/745). Regardless of which approach you take, your audit notes and the audit report should identify which documents and records you sampled and what you looked for in each document. Providing only a list of the documents is not enough detail.
Creating an auditing checklist for risk management files
Auditors with limited experience are taught to create an audit checklist by creating a table that includes each of the requirements of the audit criteria. For a risk management file, this would include a list of each of the requirements in ISO 14971 for a risk management file (i.e., Clause 9???). However, this approach is more like the approach that you should be using for a gap analysis. The better approach for creating an audit checklist for risk management files is to start by creating a turtle diagram. In the “process inputs” section (i.e., step 2 of 7), you would add questions derived from your review of the risk management plan(s). In the “process outputs” section (i.e., step 3 of 7), you would add questions specific to the risk management report and other records required in a risk management file. In the “with whom” section (i.e., step 5 of 7), you would add questions related to training and competency. You might also identify additional people involved in the risk management process, other than the process owner, to interview as a follow-up trail. In the “how done” section (i.e., step 6 of 7), you would add questions specific to the procedure and forms used for the risk management process. Finally, in the “metrics” section (i.e., 7 of 7), you would verify that the company is conducting risk management reviews and updating risk management documentation in accordance with the risk management procedure and individual risk management plan(s).
Audits are just samples
Just because you can generate a lot of questions for an audit checklist does not mean that you are required to address every question. Audits are intended to be a “spot check” to verify the effectiveness of a process. You should allocate your auditing resources based on the importance of a process and the results of previous audits. I recommend approximately three days for a full quality system audit, and approximately 90-minutes should be devoted to a process unless it is the design control process (i.e., Clause 7.3 of ISO 13485) which typically requires three to four hours due to the importance and complexity of the design controls process. Therefore, you should schedule approximately 30 minutes to interview people for the risk management process and approximately 60 minutes should be reserved for reviewing documents and records. With this limited amount of time, you will not be able to review every record or interview everyone that was involved in the risk management process. This is why auditors always remind auditees that an audit is just a sampling.
Which records are required in a risk management file?
The contents of a risk management file is specified in ISO 14971:2019, Clause 4.5. There are only four bullets in that section, but the preceding sentence says, “In addition to the requirements of other clauses of this document.” Therefore, your risk management file should address all of the requirements in ISO 14971:2019. What I recommend is a virtual risk management folder for each risk management file. As the auditor, you should also request a copy of the risk management policy and procedure. An example of what this would look like is provided below. The numbers in front of each subfolder correspond to the sub-clause or clause for that requirement in ISO 14971:2019.
Which records are most valuable when auditing risk management files?
As an auditor, I typically focus on three types of targets when auditing any process. First, I will sample any corrective actions implemented in response to previous audit findings. Second, will sample documents and records associated with any changes made to the process. Changes would also include any changes that were made to individual risk management files or the creation of a new risk management file. Finally, my third target for audit sampling is any item that I feel is at risk for safety or performance failures. The severity of the safety or performance failure is also considered when prioritizing audit sampling. In the context of a risk management file, I always verify that production and post-production activities are being conducted as planned. I try to verify that risk analysis documentation was reviewed for the need to update the documentation in response to complaints and adverse events.
More auditor training on risk management files
We are recording a live webinar intended to teach internal auditors and consultants how to perform a thorough audit of risk management files against the requirements of the new European Regulation (EU) 2017/745 and ISO 14971.
Auditing Risk Management Files
In this new webinar, you will learn how to conduct a process audit of risk management files. You will learn what to look at and what to look for in order to verify compliance with Regulation (EU) 2017/745 and ISO 14971:2019. The webinar will be approximately one hour in duration. Attendees will be invited to participate in the live webinar and receive a copy of the native slide deck. Anyone purchasing after the live event will receive a link to download the recording of the live event and the native slide deck.
In addition to this webinar on auditing risk management files, we also have other risk management training webinars available. The webinar on auditing risk management files will be hosted live on November 15, 2022 @ Noon EST (incorrect in the live video announcement).
ISO 14971:2019 includes a requirement for top management to define and document a riskmanagement policy, but do you have one?
Your risk management procedure is not your risk management policy
ISO 14971:2019 includes a requirement for a risk management policy and a risk management procedure. The word procedure is defined (Clause 3.13), a “specified way to carry out an activity or a process,” but there is no definition for policy. Both of these words begin with the letter “p,” but they are not the same. There is no guidance for a risk management policy in either of the European device regulations for CE Marking and there is no guidance in the US FDA’s regulations. In fact, there is not even a specific cause of the international risk management standard that is specific to the requirement for a risk management policy. The word “policy” only appears in ISO 14971 seven times, but the last occurrence provides the best explanation:
Appendix A2.4.2 states that “because [ISO 14971] does not define acceptable risk levels, top management is required to establish a policy on how acceptable risks will be determined.”
If someone responsible for risk management activities does not understand this distinction, this shows that risk management training may not be adequate.
Can you have a different policy for each product family?
The purpose of the policy is to establish how the acceptability of risks will be determined. However, not all devices have the same benefit-risk ratio. Therefore, if you have product families with high and low risks, then you should address this in your policy with specific criteria for each device family or create a separate risk management policy for each product family. For example, if your company is focused on designing and developing products for diabetics, you will not have the same benefit-risk profile for a Class 2 glucose reader and lancet for Type 2 diabetics that you have for an automated Class 3 insulin pumps for Type 1 diabetics. In general, separate criteria within one policy are preferred over separate policies to reduce the number of documents that must be managed.
Is there a required format for a risk management policy?
The ISO 14971:2019 standard does not include a specific format or content requirement for your risk management policy. Instead, information about the format and content of a risk management policy is provided in Annex C of ISO/TR 24971:2020. This is a guidance document, and therefore you can choose an alternate approach if you provide a justification for its equivalence. If you choose the approach recommended in Annex C, the following elements should be included:
factors and considerations for determining acceptable risk;
approaches to risk control;
requirements for approval and review.
You can download Medical Device Academy’s template for a risk management policy (POL-005) by completing the form below.
What are the factors for determining acceptable risk?
There are four possible factors to consider when determining your risk management policy:
Applicable regulatory requirements;
Relevant international standards;
And stakeholder concerns.
An example of regulatory requirements being applied to the determination of acceptable risks is the special controls defined in 21 CFR 880.5730 for insulin pumps. The special controls requirements outlined by the FDA specify design inputs as well as verification and validation requirements. The requirements are also organized into systems that comprise an insulin pump. For the digital interface requirements, the regulation specifies:
secure pairing to external devices;
secure data communication between the pump and connected devices;
sharing of state information between devices;
ensuring the pump continues to operate safely when receiving data that is outside of the boundary limits that are specified as inputs;
a detailed process and procedure for sharing pump interface specifications with connected devices.
The hazard implied by the fourth requirement above is that the pump will stop without warning or deliver the incorrect amount of insulin if the data from a continuous glucose sensor is outside of the input specifications. This design input is then addressed by a software design specification established by your company. To verify that your software risk controls are adequate, you will need to execute a verification protocol that automatically inputs a series of values that are outside of the boundary limits specified. Every time a change is made to the software, these boundary limits will need to be re-verified as part of your automated regression analysis to make sure software changes did not have an unintended effect on the device.
For software and use-related hazards, you will not be able to estimate the probability of occurrence of harm. Therefore, you shall assess the acceptability of risks based upon the severity of harm alone. Risk acceptability criteria shall be recorded in your risk management plan and the criteria shall align with your risk management policy. Ideally, these criteria are based upon international standards. For the example of an interoperable insulin pump, the following international standards are applicable:
ISO 14971, application of risk management to medical devices
IEC 62366-1, application of usability engineering to medical devices
IEC 62304, medical device software – software lifecycle processes
For the state-of-the-art, there are three examples provided in the ISO/TR 24971 guidance for how to this relates to your risk management policy:
“Leakage currents of the medical device are state of the art, demonstrated by compliance to the limits and tests regarding leakage current of IEC 60601-1.
Dose accuracy of the delivery device are state of the art, as demonstrated by compliance to the limits and tests regarding dose accuracy of ISO 11608-1.
Protection against mechanical failure caused by impact is on the same level as or better than a similar medical device, as demonstrated by comparative test such as drop test.”
Stake holder concerns is the fourth factor to consider when creating your risk management policy. Stakeholder concerns may be identified in clinical literature. However, the current trend is an emphasis on patient-reported outcome (PRO) data and post-market surveillance. Post-market surveillance is a requirement in ISO 13485, Clause 8.2.1. However, the new European MDR and IVDR have new requirements for post-market surveillance data in the technical documentation. Health Canada updated the medical device regulations to include post-market surveillance summary reports, and even the FDA is trying to develop methods for using real-world data and real-world evidence to make regulatory decisions.
Approaches to risk acceptability
The European device regulations require that a benefit/risk analysis be conducted for all risks and the overall residual risk of your device. The EU regulations also do not permit risk acceptability to consider economic impact. The EU regulations also require that risks are reduced as far as possible. Therefore, if your company is seeking CE Marking, there is only one acceptable approach suggested in ISO/TR 24971, Annex C.2: “reducing risk as far as possible without adversely affecting the benefit-risk ratio.” This is also the approach specified in our risk management procedure (SYS-010).
Requirements for review and approval of the risk policy
Requirements for approval and review of the risk management policy should be specified in the policy itself. This should specify who needs to approve that the policy is acceptable and how often the policy needs to be reviewed. Section 4.2.2 of ISO 14971 also requires that top management review the risk management process for its effectiveness. In general, we recommend that this review of the risk management process be incorporated into the management review process. Therefore, we also believe that this would be the ideal time to review the risk management policy. Generally, this is more frequently than is typically required, but if your risk management process is being reviewed for effectiveness then you have all of the necessary inputs available to review the policy as well.
This article reviews unique IVDR risk management requirements for CE Marking of in vitro diagnostic (IVD) devices in Europe.
Last week I posted a blog about “How to create an IVDR checklist.” The article was very popular because we included a form for downloading a free IVDR checklist. That form included the opportunity for people to ask a question about the IVDR. One of the subscribers, a gentleman from New Zealand, entered a very simple comment: “risk management requirements.” My first thought was that the risk management file is the required technical documentation for the IVDR. Then I quickly remembered that in 2012, EN ISO 14971:2012 was released with three new annexes for the three directives: ZA (for the MDD), ZB (for the AIMD), and ZC (for the IVDD). In Annex ZC there were seven deviations, and even though ISO 14971 was updated in 2019, the international standard continues to deviate from the European regulations in significant ways. Therefore, this blog provides an overview of the IVDR risk management requirements.
If you are already compliant with ISO 14971:2019, do you meet the IVDR risk management requirements?
The biggest difference between the ISO 14971:2019 standard and the IVDR risk management requirements is that the standard only requires a benefit-risk analysis to be performed if risks are unacceptable. In contrast, the IVDR requires that a benefit/risk analysis be performed for all risks and the overall residual risk. Therefore, you must include a benefit/risk analysis in your technical file submission regardless of risk acceptability. The harmonized version of the standard (i.e. EN ISO 14971:2019/A:11:2021) was released in December of 2021. If you already purchased ISO 14971:2019, you only need to purchase the amendment which consists of Annex ZA (comparison between the standard and Annex I of MDR) and ZB (comparison between the standard and Annex I of the IVDR).
In the amendment, it states that manufacturers must have a risk management policy that is compliant with Annex I of the EU regulation. There are notes at the beginning of each harmonization annex that indicates that the risk management process needs to be compliant with the IVDR, which means risks have to be ‘reduced as far as possible’, ‘reduced to a level as low as reasonably practicable’, ‘reduced to the lowest possible level’, ‘reduced as far as possible and appropriate’, ‘removed or reduced as far as possible’, ‘eliminated or reduced as far as possible’, ‘prevented’ or ‘minimized’, according to the wording of the corresponding section in Annex I of the IVDR. The comparison table has a column with remarks/notes. In most cases, the deficiency identified states, “Device-specific execution of the process is not covered.” There are also two remarks/notes that state “Device-specific and usability-specific execution of the process is not covered.”
Where are the IVDR risk management requirements?
IVDR risk management requirements are found in Annex II, Section 5 of the IVDR. However, there are 228 references to the word risk throughout the IVDR. The following risk-related requirements in IVDR are particularly important:
Annex I, Section 3 – reiteration of risk management procedure requirement, but specific steps in the risk management process are identified (e.g. a risk management plan, hazard identification, estimation of risks, evaluation of risks, etc.)
Annex I, Section 4 – Priority of risk control measures
Annex I, Section 5 – Elimination or reduction of use-related risks
Annex III, Section 1(b) – Reassessment of the benefit-risk analysis and risk management using post-market surveillance data
How should you document your risk management file for the IVDR?
In your risk management file of course. There is no format requirement for risk management files, but there are requirements for the content and there is a GHTF guidance document for risk management, and ISO/TR 24971:2020 is a new guidance document on the application of ISO 14971 to medical devices. Neither of these guidance documents is specific to IVDR risk management requirements. Annex H of ISO/TR 24971:2020, however, provides guidance specific to IVD devices.
What do the IVDR risk management requirements include for risk analysis?
Of these four types of risk analysis, only the software hazard analysis is sometimes not applicable. For an FDA 510(k) submission, you would need to provide software hazard analysis and URRA in the actual submission. The other two types of risk analysis would only be included in your design history file (DHF), and the FDA would review the design and process risk analysis during a routine inspection when the DHF is sampled as part of the design control process.
In contrast, the IVDR requires that a complete risk management file be submitted as part of the technical file (see Annex II, Section 5):
“Benefit-risk analysis and risk management
The documentation shall contain information on:
the benefit-risk analysis referred to in Sections 1 and 8 of Annex I, and
the solution adopted and the results of the risk management referred to in Section 3 of Annex I.”
The above documentation typically consists of design risk analysis and does not typically include process risk analysis, software hazard analysis, or use-related risk analysis. These other three risk analysis documents are IVDR risk management requirements, but they are referenced by the technical file in other sections. The most obvious IVDR risk management requirements are referenced in Annex I, Sections 1-9. These are referred to as the General Safety and Performance Requirements (GSPRs), and this requirement is typically met by including a GSPR checklist in the technical file to meet the requirement of Annex II, Section 4.
The process risk analysis is typically included with manufacturing information to meet the requirement of Annex II, Section 3.2. This documentation may include, any and all of the following elements:
a process failure mode and effects analysis (pFMEA)
a risk control plan including all processes from receiving inspection to final inspection and product release
a process validation plan that is risk-based and linked to the risk control plan
The best practice for estimation of process risks is to link the probability of occurrence and probability of detection to the quantitative data gathered during process validation. In addition, you may establish a risk management policy that prescribes specific types of process risk controls (e.g. automated inspection) for the highest risk processes where manufacturing process errors are not acceptable residual risks. For example, an inspection of printed circuit board assemblies (PCBAs) typically requires automated optical inspection (AOI) methods, because visual inspection is not sufficient by itself and not all PCBAs allow sufficient ICT coverage, and functional testing is limited.
The software hazard analysis, if applicable, is typically performed in accordance with IEC/TR 80002-1:2009, Guidance on the application of ISO 14971 to medical device software. In the software hazard analysis, it is unnecessary to estimate the probability of occurrence of harm. Instead, it is only necessary to identify hazards and estimate harm. Examples of these hazards include loss of communication, mix-up of data, loss of data, etc. Software failures are systemic in nature and the probability of occurrence cannot be determined using traditional statistical methods. Therefore, we recommend that you assume that the failure will occur and estimate software risks based on the severity of the hazard resulting from the failure. For these reasons, it is recommended that software hazard analysis documentation is maintained as a separate document from your design risk analysis. The software hazard analysis documentation should be referenced in your risk management report, but the software hazard analysis should be included as part of your software verification and validation. The IVDR requires that you include a summary of software verification and validation in Annex II, Section 6.4 rather than the complete hazard analysis document.
A use-related risk analysis should be part of your useability engineering file for IVD devices as required by EN 62366-1:2015. Use-related risks are mentioned in Annex I, Section 5:
“In eliminating or reducing risks related to use error, the manufacturer shall:
reduce as far as possible the risks related to ergonomic features of the device and the environment in which the device is intended to be used (design for patient safety), and
Give consideration to the technical knowledge, experience, education, training and use environment, where applicable, and the medical and physical conditions of intended users (design for lay, and professional, disabled or other users).”
The above requirement includes not only the ability to read and interpret test results of IVD devices but also the ability of laypersons to properly self-select if an IVD is intended to be sold as an over-the-counter product. Usability also is mentioned in Article 78, Section 3(f):
“for the identification of options to improve the usability, performance and safety of the device;”
Therefore, there should be specific elements of your post-market surveillance plan that are designed to gather feedback on the usability of your IVD device.
When should risk management activities be performed for IVD devices?
The IVDR does not specifically define when in the design and development process the various risk management activities shall be performed. However, the required risk management activities are specified in the IVDR within Annex I. ISO 14971:2019, however, is more descriptive of the risk management activities and the risk management process. Therefore, your risk management plan should align with the process defined in ISO 14971:2019, Clause 4.1.
Unfortunately, most companies do not include risk management as an integral part of the design and development process. Instead, risk management documentation is created retroactively as part of the documentation preparation for technical file submission. For this reason, most medical device executives fail to see the benefit associated with the risk management process. Even biomedical engineers struggle to appreciate the necessity of following the process outlined in the risk management standard in order to prevent device malfunctions and use errors.
The following is a list of the required risk management activities in the order that they should be occurring. Each activity also references the applicable clause of ISO 14971:2019. We have also grouped the activities into the five phases of design and development:
Risk management planning (Clause 4.4)
Identification of hazards and hazardous situations (Clause 5.4)
Design & Development
Risk estimation (Clause 5.5)
Risk evaluation (Clause 6)
Risk control option analysis (Clause 7.1)
Implementation of risk control measures (Clause 7.2)
Residual risk evaluation (Clause 7.3)
Design Verification and Validation
Benefit/risk analysis (Clause 7.4)
Risk control effectiveness verification (Clause 7.6)
Evaluation of overall residual risk (Clause 8)
Risk management review (Clause 9)
If your company is preparing a 510(k), the company may be able to submit the 510(k) immediately after completion of risk control effectiveness verification. You may also be able to postpone the benefit/risk analysis until you submit your IVD technical file for CE Marking approval. The benefit/risk analysis is not required by ISO 14971 unless the risks are unacceptable, and the FDA does not require a benefit/risk analysis except for novel devices seeking market authorization through a De Novo Classification Request or a Pre-Market Approval (PMA). The FDA also does not require the submission of the complete risk management file.
IVDR risk management requirements are quite different than the US FDA requirements for risk management. An IVD technical file must include a risk management summary report that summarizes all activities that were performed according to the risk management plan. A benefit/risk analysis is required for each risk and the overall risk. The Notified Body auditor is also expected to sample the complete risk management file during quality system audits. Finally, the IVDR includes a requirement for a post-market surveillance plan that includes the collection of production and post-production data as feedback on the risk management process and a post-market clinical performance follow-up (PMPF) plan.
What production and post-production information should you be collecting for IVD devices?
Medical device manufacturers struggle to see the benefits of requiring a post-market surveillance system, and smaller companies, in particular, complain that the cost of the new European post-market surveillance requirements is excessive and prohibits innovation. However, the primary role of post-market surveillance is to ensure rapid initiation of containment and corrective actions for devices that malfunction and/or present unacceptable risks to the intended users and intended patient population. The purpose of generating the post-market surveillance data is defined in the IVDR within Article 78, Section 3.
Information concerning serious incidents, including information from PSURs, and field safety corrective actions;
records referring to non-serious incidents and data on any undesirable side-effects;
information from trend reporting;
relevant specialist or technical literature, databases and/or registers;
information, including feedback and complaints, provided by users, distributors, and importers; and
publicly-available information about similar medical devices.
The IVDR is not prescriptive regarding what production data shall be collected for post-market surveillance, but the reason for this is that there are many different types of manufacturing processes with different process risks. In addition, the IVDR includes software as a medical device where there is no manufacturing process at all. Therefore, the best approach for determining what production data to collect is the review your process risk analysis (e.g. pFMEA). The process risk analysis for each manufacturing process should allow you to identify the manufacturing process steps that have the greatest residual risks (e.g. risk priority number or RPN) and potentially the highest severity of the effect. The risks should be identified as a priority for post-market surveillance. You should also include process parameter monitoring data for any validated processes (e.g. sterilization time, temperature, and pressure). Finally, you should also monitor rejects at incoming inspection, in-process inspection, and final inspection operations.
Other IVD Risk Management Resources
The following resources may be helpful for creating and maintaining your IVD risk management file:
You are familiar with design and process risk analysis, but do you know all four types of risk analysis?
Last week’s YouTube live streaming video answered the question, “What are the four different types of risk analysis?” Everyone in the medical device industry is familiar with ISO 14971:2019 as the standard for medical device risk management, but most of us are only familiar with two or three ways to analyze risks. Most people immediately think that this is going to be a tutorial about four different tools for risk management (e.g. FMEA, Fault Tree Analysis, HAZOP, HACCP, etc.). Instead, this article is describing the four different quality system processes that need risk analysis.
What are the four different types?
The one most people are familiar with is risk analysis associated with the design of a medical device. Do you know what the other three are? The second type is process risk management where you document your risk estimation in a process risk analysis. The third type is part of the medical device software development process, specifically a software hazard analysis. Finally, the fourth type is a Use-Related Risk Analysis (URRA) which is part of your usability engineering and human factors testing. Each type of risk analysis requires different information and there are reasons why you should not combine these into one risk management document or template.
Design Risk Analysis
Design risk analysis is the first type of risk analysis we are reviewing in this article. The most common types of design risk analysis are the design failure modes and effects analysis (dFMEA) and the fault-tree analysis (FTA). The dFMEA is referred to as a bottom-up method because you being by identifying all of the possible failure modes for each component of the medical device and you work your way backward to the resulting effects of each failure mode. In contrast, the FTA is a top-down approach, because you begin with the resulting failure and work your way down to each of the potential causes of the failure. The dFMEA is typically preferred by engineers on a development team because they designed each of the components. However, during a complaint investigation, the FTA is preferred, because you will be informed of the alleged failure of the device by the complainant, but you need to investigate the complaint to determine the cause of the failure. Regardless of which risk analysis tool is used for estimating design risks, the risk management process requires that production and post-production risks be monitored. Therefore, the dFMEA or the FTA will need to be reviewed and updated as post-market data is gathered. If a change to the risk analysis is required, it may also be necessary to update the instructions for use to include new warnings or precautions to prevent use errors.
Process Risk Analysis
Process risk analysis is the second type of risk analysis. The purpose of process risk analysis is to minimize the risk of devices being manufactured incorrectly. The most common method of analyzing risks is to use a process failure modes and effects analysis (i.e. pFMEA). This method is referred to as a bottom-up method because you begin by identifying all of the possible failure modes for each manufacturing process step. Next, the effects of the process failure are identified. After you identify the effects of failure for each process step, the severity of harm is estimated. Then the probability of occurrence of harm is estimated, and the ability to detect the failure is estimated. Each of the three estimates (i.e. Severity, Occurrence, and Detectability) are multiplied to calculate a risk priority number (RPN). The resulting RPN is used to prioritize the development of risk controls for each process step.
As risk controls are implemented, the occurrence and detectability scores estimated again. This is usually where people end the pFMEA process, but to complete one cycle of the pFMEA the risk management team should document the verification of the effectiveness of the risk controls implemented. For example, if the step of the process is sterilization then documentation of effectiveness consists of a sterilization validation report. This is the last step of one cycle in the pFMEA, but the risk management process includes monitoring production and post-production risks. Therefore, as new process failures occur the pFMEA is reviewed to determine if any adjustments are needed in the estimates for severity, occurrence, or detectability. If any of the risks increase, then additional risk controls may be necessary. This process is continuously updated with production and post-production information to ensure that process risks remain acceptable.
Software Hazard Analysis
Sofware hazard analysis is becoming more important to medical devices as physical devices are integrated with hospital information systems and with the development of software as a medical device (SaMD). Software risk analysis is typically referred to as hazard analysis because it is unnecessary to estimate the probability of occurrence of harm. Instead, it is only necessary to identify hazards and estimate harm. Examples of these hazards include loss of communication, mix-up of data, loss of data, etc. For guidance on software hazard identification, IEC/TR 80002-1:2009 is a resource. FDA software validation guidance indicates that software failures are systemic in nature and the probability of occurrence cannot be determined using traditional statistical methods. Therefore, the FDA recommends that you assume that the failure will occur and estimate software risks based on the severity of the hazard resulting from the failure.
Use-Related Risk Analysis
The fourth and final type of risk analysis is use-related risk analysis (URRA). Most development teams assume that they are able to use traditional hazard identification techniques to identify the potential use-related risks. However, use-related risks are inextricably linked to the experiences of the user. The development team has unique knowledge of the device they are developing, and therefore it is likely that use-related risks associated with a lack of knowledge about the device will result in use errors that the development team would not realize. For this reason, formative testing is necessary to identify unforeseen use-related risks. Once formative testing identifies these risks, additional formative usability testing can be used to create and refine the instructions for the use of a medical device. Finally, formative testing can be used to develop user training programs that prevent potential use errors. Once the development team has completed the necessary formative testing, then summative usability testing is used to validate the effectiveness of the risk controls that were implemented.
In the past, I believed that the FDA’s focus on usability was the review of summative usability testing. However, I have learned that the FDA feels it is equally important to begin the human factors testing process by first performing a use-related risk analysis and then identifying the critical tasks. Without identifying these critical tasks, it is not possible for the FDA to determine if the moderator of the summative testing has observed all of the critical tasks being performed correctly. An example of a Use-Related Risk Analysis (URRA) was provided by the FDA in a 510(k) AI deficiency letter that we received. The example is provided below.
Example of a URRA Table provided by the FDA
Can you use only the IFU to prevent use-related risks?
Instructions for use (IFU) are required to include warnings and precautions. This information provided by the manufacturer explains how to use a medical device correctly and identifies the residual risks. This is a form of risk control, but it is the least effective form of risk control and should be the risk control of last resort. Not everyone reads the IFU, and you cannot guarantee that everyone will understand the instructions. You certainly can’t be sure that users will remember all your warnings or precautions when they are tired, stressed, or acting in an emergency situation. Design controls and protective measures should be implemented as the first and second priority for risk controls, and the IFU should be your lowest priority.
This is the reason why we have color-coding, design features that eliminate the possibility of a use error, we provide training to users, and we are required to monitor use-related risks for medical devices. Formative usability testing is intended to identify use errors we did not anticipate, to help us develop instructions for use (IFU), and help us develop training for users. Summative testing is intended to validate that the design, training, and IFU are effective at preventing use errors. All three of these aspects work together–not the IFU alone. In fact, there is an entire alarms standard that identifies protective measures that shall be used for electromedical devices to prevent use errors (i.e. – IEC 60601-1-8).
Facilitating Risk Management Activities – An Interview with Rick Stockton
I listened to our YouTube video about the four different types of risk analysis, you may have heard my reference to Rick Stockton’s interview that we posted on our YouTube channel and embedded above. In our interview with Rick Stockton, we discussed how to facilitate risk management activities during the design and development of medical devices. If you are interested in learning more about Rick and facilitating risk management activities, please watch the video of our interview with Rick.
This blog discusses why contract manufacturers need to have a strong risk management process, and your company needs to help your contract manufacturers. This article was updated on April 28, 2022, and the original publication was January 05, 2011. Please ignore the date of publication at the top of the post for articles that are more than a year old.
Can contract manufacturers exclude risk management from the scope of their quality system?
Most contract manufacturers in the medical device industry exclude design from their Quality Management Systems. Unfortunately, most of the contract manufacturers also associate risk management with only the design process. Risk Management cannot be “not applicable” in anISO 13485Quality Management System. The requirement of section 7.1 is: “The organization shall establish documented requirements for risk management throughout product realization. Records arising from risk management shall be maintained.” The Standard also referencesISO 14971as a source of guidance on Risk Management.
Have you experienced an audit dialogue at a contract manufacturer similar to this?
The auditor asks, “How do you manage risk throughout the production process?” Then the auditee responds, “That is the responsibility of our customers. We will prepare a risk analysis if customers pay for it, but usually, customers do the risk analysis.”
For a contract manufacturer, compliance with ISO 14971 is not my primary concern as an auditor. My primary concern is to verify that contract manufacturers analyze risks associated with the processes that they perform and do their best to minimize those risks. What I don’t understand is why more companies don’t want to have strong risk management processes. Risk management is how we prevent bad things from happening. Bad stuff like scrap, complaints, and recalls. Should we expect our suppliers to have a strong risk management process?
Why your company needs to be involved in the risk management process?
Contract manufacturers should be doing everything they can to get better at risk management. During pre-production planning, they should be asking, “What happens if…” The contract manufacturer knows best HOW things will fail in production, while the customer knows best WHAT happens when things fail in production. To be safe and effective, both companies need to collaborate on risk analysis.
In any risk analysis, you need to estimate the severity of potential harm and the probability of occurrence of that harm. For production defects, the contract manufacturer can estimate the probability of occurrence of defects (i.e., P1 in Annex E of ISO 14971:2007), but the likelihood of occurrence of harm is less. The probability of occurrence of harm is the product of multiplying P1 and P2. The probability that occurrence will result in harm is P2, and P2 is a number that is less than 100% or 1. Your company can gather pre-market clinical data and post-market clinical data to estimate P2, but before launching your product, you can only guess at the value of P2. Your contract manufacturer, however, is not able to estimate P2 at all. It’s ok to estimate risk without P2 during the design phase because this will overestimate risks and result in more conservative decisions.
In addition to P2, your contract manufacturer is also not capable of estimating the severity of potential harm. As the designer of the medical device, you will know best how your device is used and what the likely clinical outcomes are when a device malfunctions. There may even be multiple possible clinical outcomes. The contract manufacturer knows what can go wrong during manufacturing, but you will need to define the clinical outcomes due to malfunctions.
Why do contract manufacturers avoid doing risk analysis?
The reason contract manufacturers avoid doing risk analysis is because it’s time-consuming and tedious.
Too bad, so sad.
Balancing my checkbook is time-consuming and tedious too, but I balance my checkbook to prevent an overdraft charge. Not doing a risk analysis can be much more painful. Scrapping out a part can cost tens or hundreds of dollars. Complaints can cost thousands of dollars. Recalls can cost millions of dollars.
If I owned a contract manufacturing company, I would ensure that everyone in the company is involved in risk management. We don’t want scrap, we can’t afford mistakes that lead to complaints, and a recall could put us out of business.
How Medical Device Academy Can Help?
Medical device academy can help both the contract manufacturer and the specification developer utilizing a contract manufacturer as a supplier! We offer training on 14971:2019 as well as procedures on risk management and supplier quality management.
Two-part Risk Management Training Webinar for ISO 14971:2019 – Part 1 of this webinar will be presented live on Tuesday, March 29 @ 9-10:30 am EDT. Part 2 of this webinar series will be presented live on Tuesday, April 5 at 9-10:30 am EDT. Purchase of this webinar series will grant the customer access to both live webinars. They will also receive the native slide decks and recording for the two webinars.
The 3rd edition of the risk management standard for medical devices, ISO 14971:2019, was released on December 16, 2019.
In October of 2018, I wrote a blog on the draft version of ISO 14971:2019 for risk management of medical devices. That article explained the differences between the different versions of the ISO 14971 standard (i.e., 2000, 2007, 2009, and 2012). I also explained what changed between ISO 14971:2007 and ISO/DIS 14971:2018. The final 2019 version of ISO 14971 3rd edition is now available.
The changes proposed in the draft included subtle changes to the names of the processes and a minor adjustment to the numbering of the clauses. Many of the annexes were also moved to ISO/TR 24971 guidance–which was released in 2020. The draft did not, however, result in a change in the overall process of risk management.
All of the changes that were discussed in my 2018 review were maintained in the final 2019 version that was released, but the ISO/TR 24971 guidance was not released at the same time as the committee had hoped for.
There are not any surprises in the 3rd edition (i.e., ISO 14971:2019). Therefore, I plan to wait until the ISO/TR 24971 guidance is released and then prepare a new blog specific to the guidance. If you are interested in training on the ISO 14971:2019 standard, the training I recorded on October 19, 2019, provides an excellent overview of these changes and highlights some of the challenges that you will encounter when trying to harmonize your risk management procedure between the ISO 14971:2019 standard and Regulation (EU) 2017/745.
This is a lot of information to absorb. Therefore, I recommend purchasing the October 2019 webinar and your copy of the ISO 14971 standard from AAMI. Anyone that has already purchased either the webinar or the procedure will receive an email offering them a discount on this new bundle that credits them for their previous purchase. If you have purchased both, you will receive credits for both purchases. Just think you can watch the video and read the new version of the standard while you are working out at the gym in January. Learn and burn!
This article explains what a pFMEA is (i.e. Process Failure Modes and Effects Analysis) and how to use them as part of your risk management process.
I recently had someone ask for help understanding the Process Failure Mode and Effect Analysis (pFMEA) a little better. I can’t blame them, because I was lost the first time I tried to fill out a form for one. It can be confusing and overwhelming if you have never created one before.
First things first, what is a pFMEA
FMEA= Failure Modes and Effects Analysis
A lower-case letter will come before the FMEA, and that denotes the ‘what’, of what the failure is that is being analyzed. A pFMEA will often be examining process failures where a dFMEA might evaluate design failures. (dFMEA’s can be confusing as well, Robert Packard created training on how to document risk management activities without using one in his Death of the dFMEA Webinar)
Some systems capitalize all the letters. Some capitalize none. That is not what is important as long as it is consistent throughout your system. Everyone should be able to easily understand that whatever variation of pfmea is used; it means “process failure modes and effects analysis.”
What does a pFMEA do?
A pFMEA will break down your manufacturing process into its individual steps and methodically examine them for potential risks or failures. For companies that utilize ourTurn-Key Quality Management System, FRM-025 process Failure Modes, and Effects Analysis can be used as a template.
For this example, we will look at receiving inspection of injection-molded casing parts for a medical device. This receiving inspection includes a manual inspection of 10 randomly selected parts out of each delivery of 100 using an optical overlay.
This area, as the section title suggests, is the process step. When looking at the process as a whole, the pFMEA will break it down into each and every step included in that process. This area is simply that individual step that is going to be examined.
The Process Step or item function depending on what your form uses for this scenario, is going to be part of the random sampling for manual inspection of the received parts using an optical overlay. Our example is going to be the backlighting element of the optical overlay display. The backlighting element will illuminate the inspected part against the template to verify that the part is within specific dimension criteria.
Potential Failure Modes
A failure mode is a way in which that process step might fail. Since it is failure modeS, it needs to be considered that there may be more than one way for the process step to fail. Do not be fooled that because this box on the form has been filled in that the pFMEA will be complete. A thorough examination of all of the possible failures should be investigated.
Our example in this process requires the backlighting element to illuminate a visual template over the parts. The light not illuminating properly is a potential failure mode of this process.
Potential Effects of Failure
the potential effects of the failure is a look into what the ramifications would be if that failure for that process step actually happened.
In our scenario, one of the potential effects of the lighting not functioning properly is that parts outside of the designated sizing acceptance criteria may be accepted rather than rejected as non-conforming parts.
The next area is the first area that requires an estimated grading of the failure. That is ‘Severity’ which is abbreviated as S. There is a scale provided in the rating section of FRM-025 that outlines the numbering system that Medical Device Academy uses.
In this case, our example is using molded plastic pieces of the outside casing of a medical device. Pieces that are too large or too small will not fit when making the final assembly of the device. These plastic pieces do not happen to be patient contacting, and do not affect the function of the device.
The evaluation of this failure is determined to have no potential effect on patient safety or increase any potential for risk of harm, therefore the severity is assigned as a ‘business risk’ meaning that it bears no risk for the user or the patient. This makes the Severity Score 0.
Causes of Failure
This column is exactly that. What might cause this identified failure to happen? In our example might be the light bulbs in the overlay machine may slowly burn out over time with use. This burnout causes potential failure.
If the bulb is expected to only have a lifetime of 100 hours, then the more hours the bulb is used, the dimmer the light may become. A slowly dimming light decreases the sharpness of the overlay template and our parts that are supposed to have a + or – size criteria of 10% now have a fuzzy template that in reality changes the overlay to show closer to + or – 13%. Now parts that are too small or too large may be accepted.
This grading criterion is also found in the Rating section of FRM-025. This is how often the failure is expected to occur. How often will the lighting element of our optical overlay fail to function in the appropriate manner for this cause?
Hopefully not very often. In fact, regularly scheduled maintenance and calibration of the overlay machine could prevent this from ever happening in the best-case scenario. Our evaluations determine that the probability of this happening is low. However, since we cannot be certain it will never happen the potential for this risk exists and makes the Occurrence score a 4.
Current Process Controls
What is currently being done to control this risk? Our example uses regularly scheduled maintenance and calibration to prevent bulb burnout affecting the overlay.
Our current process is based on routine maintenance and visual inspection. This means that the bulb burnout is something that is visually inspected for and visual inspections for detectability on the rating scale are graded as 8. This chart is found in the Rating Section of FRM-025.
RPN (Risk Priority Number)
This is a number that is found by multiplying the Severity, by the Probability, by the Detectability. In our example, the numbers RPN is 0X4X8=32 for an RPN of 32 which is considered LOW.
Below is a short video explaining the math behind calculating the Risk Priority Number
What if anything can be done to improve this process? In our example, a recommended action may be to transfer from visual only inspections to verification of light output by the meter. This makes the Detectability of the failure measurable by meter or gage which is a detectability score of 4.
This changes the RPN now to 0X4X4=16
The pFMEA shouldn’t be a solo thing
If it can be avoided this type of analysis should be done by a multidisciplinary team. Sometimes in smaller companies, people end up having to wear more than one hat. There are many entrepreneurs that have to function as the CEO/CFO/Design Engineer/RA/QA manager.
Ideally, a team approach should be used if feasible. Have the management level staff who have ownership of the processes participating in this analysis. They should know the process more intimately than anyone else in the company and should have more insight into the possible failure modes of the processes as they have likely seen them first hand. They are also the type of employee who would know the types of recommended actions to control the risk of those failures as well.
The pFMEA should also be a living document
As new failure modes are discovered they should be added to your pFMEA. A new failure mode might be discovered through a CAPA because the process had an actual failure that was not originally analyzed. Take an instance like that as an opportunity for improvement and to update your pFMEA as part of a living breathing risk management system. Also, use this as a time to re-brainstorm potentially similar failure modes that may not have been considered previously so that they can be controlled before they happen.
If you took the time to watch the video above it is also mentioned that in some instances the very first FMEA must be based on estimates because there is no data. Managers and engineers may be forced to estimate the probability of occurrence. If that is the case the FMEA should be updated in the future to adjust the (O) score to reflect what is occurring in actuality based on real data and not the theoretical data that was used for the initial estimate.
This article describes updates being made to the ISO 14971 Standard in the new draft version released for comment in July 2018.
There are two versions of ISO 14971 that are currently available. The first is the international version: ISO 14971:2007. The second is the European normative version: EN ISO 14971:2012. There is also a new draft being created by the TC210 committee for release in 2019.
Explanation of the different versions of the ISO 14971 standard
In 2000, the first edition of ISO 14971 was released as the international standard for risk management of medical devices. In 2007, the second edition of ISO 14971 was released. When new international standards are released, a European normative version is also released. The “European Norm” or EN version is intended to identify any gaps between the international standard and the requirements of the applicable European directives (i.e., the MDD, AIMD, and the IVDD). These gaps historically were included in the ZA annex at the end of the EN version. However, in 2009 this annex was split into three annexes (i.e., ZA, ZB, and ZC) to address each of the three directives separately. In reality, the 2009 annex only differed concerning the directive referenced. In 2012, a new EN version was released. This new standard included seven deviations, which were controversial. These deviations were intended to identify contradictions between the directives and the international standard, but the interpretations were not agreed with by companies or most of the Notified Bodies. Ultimately, the seven deviations were required to be addressed in the risk management files for any medical device that was CE Marked.
What changed between ISO 14971:2007 and ISO/DIS 14971:2018?
The TC210 working group assigned to update the ISO 14971 standard (JWG1) was tasked with improving guidance for the implementation of ISO 14971. Still, the committee was also tasked with making these improvements without changing the risk management process. Also, the committee was asked to move the informative annexes at the end of ISO 14971 from the standard to the guidance document ISO/TR 24971. Therefore, in July, the committee released a draft for comment and voting. Draft versions are identified with the prefix “ISO/DIS.” The ISO/DIS 14971 standard released in July has only three annexes: A) Rationale for the requirements, B) Risk management process for medical devices, and C) Fundamental risk concepts (formerly Annex E). The other seven annexes were moved to the draft of ISO/TR 24971. The reason stated for moving these Annexes to the guidance document was to make future revisions to the guidance easier to implement because it is a guidance rather than a standard. However, there were also some objectionable recommendations in the informative annexes that were the subject of deviation #3—ALARP from Annex D.8 vs. “As far as possible,” in the first indent of section 2 of Annex I in the MDD.
Although the committee was tasked to make improvements in the implementation of ISO 14971 without changing the process, the new draft has subtle changes in the process. Most of these changes can be identified quickly by reviewing the updated risk management flow chart provided in Figure 1. The updated flow chart now has two places where risks are evaluated. The first place is identical to the original Figure 1, but now the associated section is clarified to be specific to evaluating individual risks. The second place in the flow chart is new and specific to the evaluation of overall residual risks. The draft standard also states that different acceptability criteria and methods of evaluation may be used for each evaluation phase in the process. There have also been subtle changes to the names of process phases:
Section 7.4 is now “Benefit/Risk” analysis instead of “Risk/Benefit” analysis—although the draft flow chart does not reflect this.
Section 9 is now “Risk Management Review” instead of “Risk Management Report”
Section 10 is now “Production and post-production activities” instead of “Production and post-production information”
There is also more detail in the diagram under the phases for 1) risk analysis, 2) risk control, and 3) production and post-production activities.
Three new definitions are introduced in the draft standard: 3.2, benefit; 3.15, reasonably foreseeable misuse; and 3.28, state of the art. The section for identification of hazards, Clause 5.4, was reworded and expanded to consider the reasonably foreseeable sequences or combinations of events that can result in a hazardous situation. The draft standard now states that your risk management plan must also include a method to evaluate the overall residual risk and the criteria for the acceptability of the overall residual risk. In the section for risk estimation, Clause 5.5, the draft standard states that if the probability of the occurrence of harm cannot be estimated, the possible consequences shall be listed for use in the risk evaluation and risk control. The risk control option analysis priorities in section 7.1 are updated to match the new MDR, Regulation (EU) 2017/745, nearly exactly. In section 9, risk management reports were changed to risk management review, and the Clause now requires determining when to conduct subsequent reviews and when to update reports. This emphasizes the requirement to continuously update risk management documentation with input from production and post-production information. This mirrors the emphasis on continually updating post-market clinical follow-up in Regulation (EU) 2017/745, Annex XIV, Part B, Section 5, and continuously updating clinical evaluations in Regulation (EU) 2017/745, Annex XIV, Part A, Section 1.
Will ISO 14971:2019 address the 7 Deviations in EN ISO 14971:2012?
The new MDR, Regulation (EU) 2017/745, revised and clarified the wording of the essential requirements in the MDD. The MDR attempts to explain the requirements for risk management files of CE Marked products, but the MDR remains different from the requirements of ISO 14971. Unfortunately, because the ISO/DIS 14971 was not intended to change the risk management process of ISO 14971:2007, there will continue to be “deviations” between the MDR and standard.
Some people have tried to use ISO/TR 24971, the risk management guidance, as the official interpretation of how the risk management standard. However, the guidance is also a product of the TC210 committee, and it does not meet all requirements of the MDD or the MDR.
The new draft does, however, include changes that address some of the deviations in EN ISO 14971:2012. Below, each of the seven variations is listed, and hyperlinks are provided to other articles on each deviation.
Negligible Risks – The word “negligible” was only in one location in the body of the standard as a note referring to Annex D.8. In the draft, Annex D was removed and relocated to ISO/TR 24971, and the note was eliminated from Clause 3.4—now Clause 4.4 in the draft. The draft should fully resolve this deviation.
Risk Acceptability – Clause 7 was renumbered to Clause 8 in the draft. Still, the title of this Clause was also changed from “Evaluation of overall residual risk acceptability” to “Evaluation of overall residual risk.” However, if you read the Clause it still refers to determining the acceptability of risks. In note 2 of Annex ZA of the draft, it states that determining acceptable risk must comply with Essential Requirements 1, 2, 5, 6, 7, 8, 9, 11, and 12 of the Directive. The draft should fully resolve this deviation.
ALARP vs. “As far as possible” – The European Commission believes that the concept of “ALARP” implies economic considerations, and some companies have used economics as a reason for not implementing certain risk controls. ALARP was eliminated from the notes in the risk management plan clause and by moving Annex D.8 to ISO/TR 24971 and adding note 1 in Annex ZA. The draft should fully resolve this deviation.
Benefit/Risk Analysis – The contradiction in requirements between the International Standard and the MDD, as it relates to determining when a benefit/risk analysis must be conducted, has not been updated. The draft does not resolve this deviation. Companies that CE Mark products will need to perform a benefit/risk analysis for all residual risks and all individual risks—despite the wording of the standard.
Risk Control – The contradiction in requirements between the International Standard and the MDD, as it relates to determining when risk controls must be implemented. The International Standard gives companies the option to avoid the implementation of risk controls if the risk is acceptable. At the same time, the MDD requires that risk controls be implemented for all risks unless the risk controls create additional risks that increase risks, or the risk controls do not reduce risks further. The draft does not resolve this deviation. Companies that CE Mark products will need to implement risk controls for all individual risks—despite the wording of the standard.
Risk Control Options – The intent of Clause 6.2 in ISO 14971:2007 was likely to be the same as the MDD. However, the European Commission identified the missing word “construction” as being significant. Therefore, to prevent any misunderstandings, the TC210 committee copied the wording of Regulation (EU) 2017/745. The draft should fully resolve this deviation.
IFU Validation – Again, to prevent any misunderstandings, the TC210 committee copied the wording of Regulation (EU) 2017/745. However, the examples of information for safety (i.e., warnings, precautions, and contraindications) were not included. Hopefully, the final version of the 3rd edition will consist of these examples. Clause 8, evaluation of overall residual risk, was also reworded to state, “the manufacturer shall decide which residual risks to disclose and what information is necessary to include in the accompanying documentation to disclose those residual risks.” The draft should fully resolve this deviation.
Recommendations for your Risk Management Process?
The most important consideration when establishing a risk management process for medical devices is whether you plan to CE Mark products. If you intend to CE Mark products, then you should write a procedure that is compliant with the current requirements of the MDD and future requirements of Regulation (EU) 2017/745. Therefore, the seven deviations should be addressed. Also, you need to maintain compliance with the current version of the standard.
I recommend creating a process based upon the newly updated process diagram in the latest draft. The process should begin with a risk management plan. For your plan, you may want to create a template and maintain it as a controlled document. It could also be part of your design and development plan template, but the plan should include each of the following risk management activities:
Risk control option analysis
Risk control verification of effectiveness
Evaluation of overall residual risk
Risk management review
Production and post-production activities
Your procedure should also be integrated with other processes, such as 1) design control, 2) post-marketing surveillance, and 3) clinical evaluation. Your procedure must indicate the priority for the implementation of risk control options. The best strategy for ensuring risk control priorities are compliant is to copy the wording of the new EU Regulations verbatim. Your process should include performing benefit/risk analysis. You should also define your process for risk management review. Your review process should specify when subsequent reviews will be done, and when your risk management report will be updated. Finally, you should identify a post-market surveillance plan for each device or device family, and use that post-market surveillance data as feedback in the risk management process.
The one element that appears to be weakly addressed in the body of the standard is the requirement for traceability of each hazard to the other aspects of the risk management process. Although traceability is mentioned in Clause 3.5 of the 2nd edition, and Clause 4.5 of the draft 3rd edition of ISO 14971, that is the only place is mentioned in the body of the standard. Traceability is mentioned several more times in Annex A, but the focus seems to be on the risk management file. Companies need more guidance on how to achieve this traceability. The appropriate place for this guidance is probably in ISO/TR 24971. Still, in order to maintain this documentation, a software database will likely be critical to maintaining traceability as changes are made during design iterations and after commercialization. This type of software tool is also needed to expedite the review of risk management documentation during a complaint investigation.
Which Risk Analysis Tool should you use?
In Annex G of ISO 14971:2007 and the EN 2012 version, there are five different risk analysis tools described. The word “described” is emphasized because informative annexes are not “recommended.” The committee that created the 2nd edition of ISO 14971 wanted to provide several suggestions for possible risk analysis tools to consider. However, each tool has strengths and weaknesses. Additionally, the widespread use of the failure-mode-and-effects analysis (FMEA) tool in the automotive and aerospace industries has spread to the medical device industry, and companies seem to believe that regulators prefer the FMEA tool. This is not true. Companies should be trained in all of these tools, and training should consist of more than just reading Annex G, and the tools should be used where they are most beneficial. My recommendations are below:
Preliminary Hazard Analysis (PHA) – This process is critical during the development of design inputs. It is also the most underutilized analysis tool. I have not seen a single example of this tool written in a procedure by any medical device company. I believe this process should be continuously updated as part of training new design team members and should be both product and project-specific.
Fault-tree Analysis (FTA) – This process is a top-down approach to risk analysis. It is heavily utilized by transportation engineers when intersections are designed, and accidents are investigated. This tool depicts risk analysis pictorial as a tree of fault modes representing each possible root cause for failure. At each level of the tree, fault mode combinations are described with logical operators (i.e., AND, OR). The information displays the frequency of each fault mode quantitatively. Therefore, when you are investigating a complaint, the tree can be used to help identify possible fault modes that may have been the root cause of device failure. You may also be interested in the standard specific to Fault tree analysis (FTA): IEC 61025:2006.
Failure Mode and Effects Analysis (FMEA) – This process is a bottom-up approach to risk analysis. The automotive and aerospace industries heavily utilize it. This tool systematically lists all failure modes in groups organized by component. Risks are estimated based upon the severity of effect, probability of occurrence, and detectability. Over time, the FMEA process split into three tools: 1) process FMEA (pFMEA), 2) design FMEA (dFMEA), and 3) use FMEA (uFMEA). The first is ideal for analyzing and reducing risks associated with the manufacturing of devices. In particular, the detectability factor can be linked closely with process validation. The second evolved from the realization that the detection of a risk after the device is in the user’s hands does not reduce risk. A risk reduction only occurs if detectability is proactive. Therefore, this was stated in Annex G.4, and companies began to eliminate detectability and continued to use FMEA as their primary tool. Due to the widespread familiarity with the FMEA tool, usability FMEAs became popular for documenting risks associated with the use of a device. Unfortunately, the only real advantages of a dFMEA and uFMEA are familiarity with the tool. You may also be interested in the standard specific to FMEA: IEC 60812:2018.
Hazard and Operability Study (HAZOP) – In addition to the risks of using devices, there are also risks associated with the production of devices. Processes related to coating, cleaning, and sterilization are all processes that typically involve hazardous chemicals. The chemical and pharmaceutical industries use HAZAP as a tool to analyze these process risks and prevent injuries. You may also be interested in the standard specific to HAZOP: IEC 61882:2016.
Hazard Analysis and Critical Control Point (HACCP) – This process is primarily used by the food industry to prevent the spread of contaminated food supplies. Even though medical device manufacturers do not typically use it, it should be considered as a tool for managing the supply chain for devices. This model is useful when manufacturing is outsourced, or secondary processing is conducted at second and third-party suppliers. Since many FDA inspectors started in the food industry as inspectors, this is also a method that is supported by the FDA as a risk control process for outsourced processes.
How to document your risks?
For simple devices, risk management documentation is a burdensome task. For complex devices, a spreadsheet could include hundreds of lines or more than even one thousand individual lines. Also, the requirement for traceability requires additional columns in a table. Therefore, it becomes nearly impossible for you to include all the required information on a page that is 11 inches wide. If you expand your page to 17 inches wide, the size of your font will need to be very small. If you make a change, your spreadsheet can be challenging to update quickly. You could purchase a 43” widescreen TV for your monitor, or you can use dual monitors for your display, but changes remain challenging to implement without a mistake.
You need to stop relying upon spreadsheets. Use a database, and don’t use Microsoft Access. Purchase a database that is designed to document design controls and risk management traceability. If your company has software expertise, develop your software tool to do this. You should also design standardized templates for exporting your reports. By doing this, it will only take minutes to create an updated report when you make design changes. If you describe the risk management activities as notes in your software, the description of these activities can also be automatically converted into summary pages for each report summarizing that risk management activity. You can even prompt the user to answer questions in the software to populate a templated document. For example, you can prompt users to input subsequent updates of your risk management reviews, and that can be automatically converted into a summary paragraph. This reporting capability is especially helpful when responding to FDA review questions asking for cybersecurity risks.
Additional Training Resources for ISO 14971
The risk management training webinar has been completely rewritten for the second time (i.e. the first time was on October 19, 2018). The newest version will be a two-part webinar series. Part one of two will focus on Clauses 1 through 7.1 of the ISO 14971:2019 standard. Part two of two will focus on Clauses 7.2 through 10. We selected Clause 7.2 to begin the second part of this webinar series, because it marks the beginning of the verification of the risk controls your company has implemented (i.e. – Post “Design Freeze”). Part 1 will be hosted live on March 29, 2022 @ 9-10:30 am EDT, and Part 2 will be hosted live on April 5, 2022 @ 9-10:30 am EDT. Both sessions will be recorded if you are unable to participate in the live sessions.
SYS-010, Medical Device Academy’s Risk Management Procedure, is compliant with EN ISO 14971:2019. The procedure includes templates for documentation of design risk management and process risk management. The procedure is also compliant with ISO/TR 24971:2020 and Regulation (EU) 2017/745. Both the two-part risk management training webinar, and the risk management procedure, are included in Medical Device Academy’s turnkey quality system.
What are the differences between 510k risk management requirements and risk management requirements for your Design History File (DHF)?
Risk management requirements integration with design
Last week I presented a free webinar on how to combine risk management with design controls when planning to submit a 510k. Many questions were asking what the design control and risk management requirements are for a 510k.
What are the 510k design control requirements?
There is no specific part of the regulations stating what the 510k design control requirements are. However, some aspects of the DHF are required as 510k design control documentation, but not necessarily in the exact form as maintained in the DHF. For example, Design Inputs and Design Outputs are presented as applicable recognized standards and design specifications, while others will remain precisely the same (i.e., verification and validation test reports).
What are the Risk Management Requirements in a 510k?
For 510k submissions, the only risk management requirements are the inclusion of risk documentation for devices containing software of at least moderate level risk. There are some exceptions to this as well, though, based on a few special control guidance documents—especially when the submission type is an abbreviated 510k. This is article identifies which of the DHF and RMF elements are 510k design control requirements and 510k risk management requirements.
Quality system requirements for design controls
Design Controls are identified in 21 CFR 820.30. Every manufacturer of any Class II or Class III devices and certain Class I devices (Class I devices with software, tracheobronchial suction catheters, surgeon gloves, protective restraints, radionuclide applicators, radionuclide teletherapy devices) need to control design per this regulation. The requirement for a Design History File is item j) and states:
“Each manufacturer shall establish and maintain a DHF for each type of device. The DHF shall contain or reference the records necessary to demonstrate that the design was developed following the approved design plan and the requirements of this part.”
The “requirements of this part” refer to the other bullets in 21 CFR 820.30 which can be summarized as:
a) Establish and maintain procedures to control the design of a device.
b) Design and Development Planning – Each manufacturer shall establish a plan that describes the design and development activities and defines responsibilities for implementation.
c) Design Inputs – Manufacturers need to ensure design requirements relating to a device are appropriate and address the intended use of the device.
d) Design Outputs – Design outputs need to be documented in terms that allow an adequate evaluation of conformance to design input requirements. Design outputs that are essential for the proper functioning of the device should be identified.
e) Design Review – Formal documented reviews of design results should be planned and conducted at appropriate stages of device development.
f) Design Verification – Design verification confirms that the design output meets the design input requirements.
g) Design Validation – Design validation shall be performed under defined operating conditions on initial production units or their equivalents. It shall ensure that devices conform to defined user needs and meet the intended use of the device.
h) Design Transfer – Design transfer documentation shall ensure that the device design is correctly translated into production specifications.
i) Design Changes – changes should be identified, documented, validated/verified, reviewed, and approved before their implementation.
The Design History File is intended to be a repository of the records required to demonstrate compliance with your design plan and design control procedures. While companies are required to create and maintain this documentation according to the FDA regulation, not all of the documentation will be reviewed as part of the 510k. The following table compares the elements that comprise a DHF with the 510k design control requirements.
510k Design Control Requirements
User Needs & Design Inputs
Declaration of Conformity
User needs are design requirements that require design validation (e.g., adequacy of user training, and safety/performance of the device for the indications for use). Some design inputs will appear in the form of standards in the FDA eSTAR template. If you are declaring conformity with these standards, a Declaration of Conformity is automatically created in the FDA eSTAR template.
Device Description (Section 11)
The Device Description lists the specifications of the device, and your Design Outputs document will help populate the Device Description. This can include drawings, pictures, or written specifications that describe your device.
Proposed Labeling (Section 13)
The labeling is usually considered part of the Design Outputs within the DHF and is included specifically in the labeling section of the 510(k) submission. This includes both the Instructions for Use and any Package Labeling.
Verification and Validation Protocols
You do not have to include the protocols, but the reviewer may ask to see them if they have any questions when reviewing the reports.
Verification and Validation Reports
Sterilization (Section 14)
Biocompatibility (Section 15)
Software (Section 16)
Electrical Safety and EMC (Section 17)
Bench Performance Testing (Section 18)
Animal Performance Testing (Section 19)
Clinical Performance Testing (Section 20)
Of course, not all of these sections will be applicable to every device. Still, you should include all relevant validation test reports within your submission in the appropriate part of the 510k. Typically, each of these sections will have a cover sheet that outlines the reports that are included within the section, and then you can just include the report from the DHF in its entirety behind the cover sheet in that section.
Only required for sterilization validation typically, but there are exceptions for novel materials and coatings
Not Required for 510k
Design Review Meeting Minutes
Not Required for 510k
Design Trace Matrix
Only required for software
Risk Management File
Sometimes – See Risk Management File Table Below
Post-Market Surveillance Plan
Not Required, but a few exceptions for high-risk devices
Clinical Data Summary
Required only if used to demonstrate safety and efficacy
It Will result from 510k Clearance, so nothing is to be included in the 510k submission.
510k Risk Management Requirements
Regarding the FDA regulations for risk management, there is a requirement under the Design Validation section of 21 CFR 820.30 that states:
“Design validation shall include software validation and risk analysis, where appropriate.”
For FDA compliance and CE Marking, both recognize ISO 14971 as the standard for risk management. FDA recognizes ISO 14971:2007 whereas EN ISO 14971:2012 is the European National version for CE Marking. Rob Packard wrote an article describing the contents of the risk management file as well as the specific differences in the requirements between the FDA and CE Marking with regard to ISO 14971.
For your 510k submission, the FDA only requires risk management documentation to be included if the product contains software, and the risk is at least a level of “moderate concern”. There are some other cases when risk management is required by special controls guidance documents, but even when it is required, you only have to submit your risk analysis. The table below describes the risk management requirements in greater detail.
510k Risk Management Requirement
Risk Management Plan
510ks with Software Only (Section 16)
Hazard Identification is only required for devices that have a software component. It is not required for most other devices.
510(k)s with Software (Section 16)
Certain Special Controls Guidance
The Risk Assessment is only required to be included in your device contains software, or if a special controls guidance document specifically requires a risk assessment. It is not required for other 510ks.
Risk Control Option Analysis
Software and Certain Special Controls Guidance
Risk Control Verification and Validation
Sterilization (Section 14)
Biocompatibility (Section 15)
Software (Section 16)
Electrical Safety and EMC (Section 17)
Bench Performance Testing (Section 18)
Animal Performance Testing (Section 19)
Clinical Performance Testing (Section 20)
This will not be any additional or special documentation specific to Risk Management and was already included in the DHF breakdown above. Still, the verification and validation also relate to risk management in ensuring that the risks have been adequately mitigated.
Not Required for 510(k)
Risk-Benefit analyses are only required for De Novo applications, Humanitarian Device Exemptions, and PMAs.
Informing Users and Patients of the Risks
Labeling (Section 13)
Part of the risk management will appear in the Labeling section of the 510k as warnings, contraindications, and precautions within the Instructions for Use and Package Labeling.
Risk Management Report
Special Controls Guidance Documents with Risk Management Requirements
Your first step in preparing your 510k submission is to search the FDA Guidance Document Database to determine if there is an applicable guidance document for your device. You can read another blog we wrote to explain Special Controls Guidance documents, and how to determine if one applies to your device. The following list provides examples of Class II Special Controls Guidance documents that require risk analysis to be included within the 510k:
When there are 510k risk management requirements, the special controls guidance document will typically state, “We recommend that the summary report contain:
An identification of the Risk Analysis method(s) used to assess the risk profile in general as well as the specific device’s design and the results of this analysis. (Refer to Section 6 for the risks to health generally associated with the use of this device that the FDA has identified.)
Discussion of the device characteristics that address the risks identified in this class II special controls guidance document, as well as any additional risks identified in your risk analysis.”
The special controls guidance will also identify risks to health that have been identified for products of that type, which you should be sure to include in your risk analysis as appropriate.
More Information on Design Control and Risk Management Requirements
Hopefully, you are now able to determine which elements of your DHF are 510k design control requirements and which elements of your RMF are 510k risk management requirements. If you would like more information about how to implement design controls and risk management within your product development process, please consider registering for one of our training webinars:
If you need any further information or specific assistance with your 510k submission, please feel free to send me an email at email@example.com or schedule a call with our principal consultant, Rob Packard. He can answer any of your medical device regulatory questions.