Risk Management Medical Device Academy

Archive for ISO 14971:2019 (Risk Management)

Auditing ISO 14971 – 4 Steps to Assess Compliance

Posted by Rob Packard on December 2, 2012

This article describes four key steps for auditing ISO 14971, and suggested auditing questions are included.

Let’s say that you went ahead and purchased ISO 14971:2012, read Annex ZA, and identified a couple of gaps in your procedure. After you revised your Risk Management procedure to be compliant with the revised Standard, then what are you supposed to do?

Most QA Managers struggle over whether they should purchase ISO 14971:2012. I wrote a couple of blog postings about this matter, but my point was not to debate this question but to ensure companies are aware that they need to be compliant with the MDD and the ISO 14971 Standard. The “changes” from 2009 to the 2012 version are simply the European Commission reminding manufacturers that there are seven aspects of the ISO 14791 Standard that do not meet the requirements of the MDD. Therefore, if your company has already verified that your risk management process is compliant with the MDD–then you have nothing to change. However, if your risk management process is only compliant with ISO 14971:2009, then you need to revise your processes and procedures to address these seven aspects.

4 Steps in Auditing ISO 14971

Once you have made revisions to your risk management process, how do you perform auditing of ISO 14971?

Step 1: Planning your auditing ISO 14971

This will be an internal audit, and since you (the QA Manager) are the process owner for the risk management process, you personally cannot audit this process. You need to assign someone that has the technical skill to perform the audit, but this person cannot be the process owner (you) or a direct report to the process owner (the rest of the QA department). Fortunately, the Director of Engineering is also trained as an internal auditor at your company. She is trained on ISO 14971:2009, but she did not receive risk management training to the most current version. To address this gap, she must read the updated Standard to understand what’s new.

novcover preview 211x300 Auditing ISO 14971 4 Steps to Assess Compliance

Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

She has participated in risk management activities, but each product development engineer participates in risk management activities for their own design projects. Therefore, she has several projects she can sample risk management records from without auditing her own work. You have communicated that you need this audit finished sometime in December because you want any CAPAs resulting from the audit to be finalized before the next Management Review at the end of January. The timing of the Management Review is important because the risk management procedure requires that top management assess the effectiveness of the risk management process during Management Review meetings.

There are no previous audit findings to close from the last audit of the risk management process. Still, the Director of Engineering has seven specific items to emphasize from the 2012 revision of the Standard, and a revised procedure for risk management. Therefore, she will prepare for the audit by identifying some new interview questions to specifically address these changes–as well as some more general, open-ended questions.

Specific questions related to Annex ZA when auditing ISO 14971

1. How does the risk analysis evaluate the acceptability of risks in the lowest category? (This is a leading question, but it is specifically designed to determine if negligible risks are discarded).

2. Please provide a few examples of how risks in the lowest category were reduced. (In sections 1 and 2 of the Annex, I require all risks to be reduced as far as possible, and for all risks to be evaluated for acceptability. The wording of this question also allows auditors flexibility in their sampling).

3. How did the design team determine when they had implemented sufficient risk controls to minimize risks? (Many companies use a color-coded matrix as a quasi-objective method for determining when risks are adequately reduced. This process is often referred to as the ALARP concept. Annex ZA specifically prohibits using economic considerations as part of this determination).

4. How did you conduct a risk-benefit analysis? (The Standard allows for performing a risk-benefit analysis when overall residual risks exceed the acceptability criteria as outlined in the risk management plan. However, the MDD requires an overall risk-benefit analysis in Section 1 of Annex I. Section 6 also requires that a risk-benefit analysis be performed for each individual risk).

5. How were risk control options selected? (Section 2 of the MDD implies that the manufacturer shall review All the control options and pick the most appropriate ones. Therefore, the auditor should specifically look for evidence that the team systematically reviewed all possible control options to reduce risks–rather than stopping as soon as the risks were reduced to an acceptable level).

6. What were your team’s priorities for the implementation of risk control options? (It’s possible that the previous question will be sufficient to gather evidence that risk controls were implemented with the required prioritization, as specified in the MDD. However, this question would be used as a follow-up question if it is not clear that the team prioritized the risk control options in accordance with Section 2 of Annex I).

7. How was the effect of labeling and warnings in the instructions for use incorporated into the estimation of residual risks? (Almost every company remembers to include residual risks in their IFU as a warning or caution statement. However, Section 2 of Annex I does not allow for including this information given to the users as a method of reducing risks. Therefore, in a Design FMEA, you would not list labeling and IFUs in your column for current risk controls when you determine the risk. This should be identified as an action to be taken–with no impact on the score for residual risk).

The above questions are not examples of using the process approach, but each question is phrased in an open-ended manner to maximize the objective evidence gathered during the interview process. If you are doing a process audit, it’s still acceptable to include questions that use the element approach.

Generic questions when auditing ISO 14971

1. When was the ISO 14971:2012 version of the Standard added to the controlled list of external Standards?

2. Please provide examples of where you have updated the Essential Requirements Checklist (a Technical File document) to reference the newest revision of ISO 14971:2012, and please show at least one example of how the risk management report was updated to reflect this revision.

3. How did you verify training effectiveness for the design team specific to the updated risk management procedure before conducting a risk analysis?

These generic questions do not require reading the ISO 14971:2012 Standard. Instead, each question forces the auditee to demonstrate their knowledge of the revised Standard by answering open-ended interview questions. Each of these questions is also designed to test linkages with other support processes. This is an example of how to use the process approach.

Step 2: Auditing ISO 14971

The next step is to conduct your audit of ISO 14971. During the auditing of ISO 14971, the Director of Engineering will gather objective evidence of both conformity and nonconformity for the risk management process. The generic interview questions that were developed allow her to evaluate the effectiveness of linkages between the risk management process and other processes, such as:

1) Document control

2) Creating technical documentation for regulatory submissions

3) The training process

Specific questions verify that each of the seven elements identified in Annex ZA of ISO 14971:2012 is adequately addressed in the revised procedure. When the audit is completed, the auditor will have a closing meeting with the process owner (you) and the auditee(s), so that everyone is clear about what the findings were, and if there were any nonconformities. This is the time to clarify what needs to be done to prevent each nonconformity from recurring.

Step 3: Writing the Report & Taking Corrective Action(s)

This is no different from any other audit. Still, it is critical to have the report completed soon enough so that CAPAs can be initiated (not necessarily completed) before the Management Review.

Step 4: Verifying Effectiveness of Corrective Action(s)

Many people struggle with verifying the effectiveness of corrective actions–regardless of the process. My advice is to identify a process metric to measure effectiveness. Then the effectiveness check is objective. For example, monitoring the frequency of updates to the list of external standards can help verify that the process for monitoring when Standards are updated is effective. Likewise, the frequency of updates to the Essential Requirements Checklist and the risk management records referenced in the Essential Requirements Checklist indicates if the risk management process is being maintained. Finally, monitoring the lag between the time procedures are updated and when the associated training records are updated quickly identifies if there is a systemic problem with training or if a training gap is just an example of a single lapse.

Tags: Auditing ISO 14971:2012, ISO 14971:2012, ISO 14971:2012 compliance

Posted in: ISO 14971:2019 (Risk Management)

Do you need to purchase the latest EN ISO 14971 version?

Posted by Rob Packard on August 1, 2012

It is not necessary to purchase the EN ISO 14971 version because you should already be compliant and amendments are sold separately.

Discussion about a risk management standard 1024x664 Do you need to purchase the latest EN ISO 14971 version?

If the above conversation sounds familiar, hopefully, this blog will help.

Note: This is a 2012 blog that will be updated and/or consolidated soon, but here’s a link for risk management training.

Question 1: What is the current version of EN ISO 14971?

Answer 1: EN 14971 was revised to 2012 on July 6, 2012. The previous 2009 version was withdrawn. The ISO version is not changing–just the EN version.

Question 2: What’s new in 2012?

Answer 2: Only the three Annexes related to harmonization with the three directives (MDD, AIMDD, and IVDD) were updated. The content of the Standard itself has not changed.

Question 3: Do I need to buy EN ISO 14971… which really hasn’t changed since 2007?

Answer 3: No…unless you still have the 2000 version. (just my personal opinion … not anyone else necessarily agrees)

Why you don’t need to buy the EN ISO 14971 version…

Historically, Annex ZA was the annex at the back of a Standard that would explain how it is harmonized with the European Directives. However, in 2009, Annex ZA was separated into ZA, ZB, and ZC. Each of these Annexes explained how the current version of ISO 14971 (then ISO 14971:2007) differs from each of the three directives. In addition, there was a correction to Figure 1 (i.e., – arrow in the wrong location). Neville Clarke provided a good summary of these minor changes that occurred in 2009. The European Commission was concerned with some of the differences between the 2009 Standard and the Directives. Therefore, the Standard has been updated to clarify these differences.

There are seven technical deviations from the Standard that are required for compliance with the European Directives. Marcelo Antunes is an expert on Standards, and he accurately describes these deviations as “weird” in a discussion thread on Elsmar Cove’s Forum. The deviation that seems to have caught the most attention is the requirement to reduce ALL risk to “as low as possible” (ALAP) rather than to a level that to “as low as reasonably practicable” (ALARP concept). The “ALAP” acronym was a joke, but it wouldn’t be the first time that something like this stuck (i.e., – SWAG).

An alternative approach to verifying compliance with EN ISO 14971

If you sleep with a label maker under your pillow, you should buy the new BS EN 14971:2012 version, so you can ensure that you are staying in compliance with each of these seven deviations and that you have considered the implications fully in your procedure for Risk Management. However, if you are a practical person that prefers not to upset the entire development team, I recommend a different approach.

1. Download a copy of the relevant Directive from the Europa Website

2. Using Adobe, search the entire Directive for the word “risk”:

AIMDD = 24 times

MDD = 55 times

IVDD = 34 times

3. Systematically review where the word “risk” is used to determine if you need to make adjustments for your CE Marked products. If you already have a CE Mark, there should be no changes required to your risk management documents. Your procedures might need clarification to observe the requirements of the Directive when there is a difference between the Standard and the Directive.

Last Question: What is your Notified Body auditor going to do?

Final Answer: I’m not sure, because every auditor is a little different in their approach. However, as an instructor, I would teach an auditor to ask open-ended questions, such as: “How did you determine if there is an impact upon your procedures and design documentation with regard to the updated Standard?” (i.e., – impact analysis). If the company provides an impact analysis and explains why the existing risk documentation and procedure should not change, I believe this meets the requirements for “equivalency with the State of the Art.”

Honestly, I haven’t seen one single company that was 100% in compliance with the “letter” of the Directives or the Standard. Sometimes, rational thought must overcome political compromises and irrational behaviors.

On the other hand, it’s always possible that these seven deviations, and the information on corrective action, will fundamentally change the way your company approaches risk management (I just dare you to bring it up in your next management review).

If you would like a second opinion, the Document Center’s Standard Forum says, “As you can see, this material is essential to conformance with the EN requirements and will make the purchase of the EN edition (BS EN ISO 14971 is the official English language edition) mandatory for medical device manufacturers certifying to the standard for sales in Europe.” FYI…Document Center’s Standard Forum sells Standards. You can buy this one from them for $324.

Tags: EN 14971:2012, ISO 14971

Posted in: ISO 14971:2019 (Risk Management)

Learning Pyramid – 4 Levels of Learning

Posted by Rob Packard on April 2, 2011

The author discusses the four levels of learning in the Learning Pyramid, and the lessons learned when he taught an ISO 14971 Risk Management course.

I am in Canada, it’s almost midnight, and my client has me thinking so hard that I can’t sleep. I am here to teach the company’s Canadian facility about ISO 14971:2007—the ISO Standard for Risk Management of medical devices.

Most of the companies that request this training are doing so for one of two reasons: 1) several of their design engineers know almost nothing about risk management, or 2) they have several design engineers that are quite knowledgeable concerning risk management, but these engineers have not maintained their credentials, and their last risk management training was related to the 2000 version of the Standard. This company falls into the second category.

I always tell students that I learn something by teaching each course. From this company, however, I have learned so much. This company has forced me to re-read the Standard several times and reflect on the nuances of almost every single phrase. I have learned more about this Standard in one month than I learned in the 3.5 years since I first took the course I am now teaching.

The four levels of the Learning Pyramid

I have developed a model for learning that explains this phenomenon. I call this model the “Learning Pyramid.” At the base of the pyramid, there are “Newbies.”

This is the first of four levels. At the base, students read policies and procedures with the hope of understanding.

In the second level of the pyramid, the student is now asked to watch someone else demonstrate proper procedures. One of my former colleagues has a saying that explains the purpose of this process well, “A picture tells a thousand words, but a demonstration is like a thousand pictures.” This is what our children call “sharing time,” but everyone over 40 remembers this as “show and tell.”

In the third level of the pyramid, the student is now asked to perform the tasks they are learning. This is described as “doing,” but in my auditing courses, I refer to this process as “shadowing.” Trainees will first read the procedures for Internal Auditing (level 1). Next, trainees will shadow the trainer during an audit as a demonstration of the proper technique (level 2). During subsequent audits, the trainees will audit, and the trainer will shadow the trainee (level 3). During this “doing” phase, the trainer must watch, listen, and wait for what I call the “Teachable Moment.” This is a moment when the trainee makes a mistake, and you can use this mistake as an opportunity to demonstrate a difficult subject.

Finally, in the fourth level of the Learning Pyramid, we now allow the trainee to become a trainer. This is where I am at—so I thought. I am an instructor, but I am still learning. I am learning what I don’t know.

Teaching forces you back to the bottom of the Learning Pyramid

The next step in the learning process is to return to the first level. I am re-reading the Standard and procedures until I understand the nuances that I was unaware of. Then, I will search for examples in the real world that demonstrate these complex concepts I am learning. After searching for examples, I will test my knowledge by attempting to apply the newly acquired knowledge to a 510(k) or CE Marking project for a medical device client. Finally, I will be prepared to teach again.

This reiterative process reminds me of the game Chutes and Ladders, but one key difference is that we never really reach the level of “Guru.” We continue to improve, but never reach our goal of perfection…For further inspiration, try reading “Toyota Under Fire.”

Tags: ISO 14971, Training

Posted in: Education, ISO 14971:2019 (Risk Management)