Blog

Archive for Risk Management

Risk Acceptability – Deviation #2 in EN ISO 14971

This 7-part blog series continues with the author reviewing deviation #2, risk acceptability, in the EN ISO 14971:2012 Standard.
%name Risk Acceptability   Deviation #2 in EN ISO 14971

In 2012, the European National (EN) version of the Medical Device Risk Management Standard was revised, but there was no change to the content of Clauses 1 through 9. Instead, the European Commission identified seven content deviations between the 14971 Standard and the requirements of three device directives for Europe. This seven-part blog series reviews each of these changes individually. The second deviation is specific to risk acceptability.

Discretionary power of manufacturers as to Risk Acceptability

The second deviation is specific to determining risk acceptability in the risk evaluation process. The ISO 14971 Standard indicates in Annex D4 that the acceptability of risk is not specified by the Standard and must be determined by the manufacturer. In Clause 3.2 of the 14971 Standard, it states that, “Top management  shall: define and document the policy for determining criteria for risk acceptability.” This risk management policy is intended to indicate a threshold for risk acceptability. In Clause 5 of the 14971 Standard, the manufacturer is instructed to evaluate whether risks are acceptable using the risk management criteria defined in the risk management policy.

Essential requirements 1 and 2 require that risks be reduced as far as possible, and that all risks shall be included in a risk-benefit analysis—not just the risks that exceed a certain threshold for risk acceptability. Therefore, the requirement to establish a risk policy for the acceptability of risk directly contradicts the MDD.

Since the 2nd edition of the 14971 Standard was first issued (i.e., -2007), clients have been asking me how to establish a risk acceptability criteria. For new devices, I recommend benchmarking the risks of the new device against existing devices. In other words, if the new device presents equal or lower risks than existing devices, then the risks of the new device are acceptable. For existing devices, I recommend performing a risk-benefit analysis, evaluating adverse events observed with the device against the benefits of using the device. Unfortunately, most companies choose arbitrary thresholds for risk acceptability. Instead of relying upon benchmarking or risk-benefit analysis, companies will establish a policy that all risks must be below a quantitative value. For example, if the range of possible risks scores are from 1 to 1,000, all risks of 100 or less may be acceptable.

What is acceptable?

In order to comply with the EN ISO 14971:2012 version of the risk management standard, you will need to implement risk controls for all risks, regardless of acceptability. However, you will also need to perform a risk-benefit analysis. The risk-benefit analysis should consider not only the benefits to patients and the risks of using the device, but the analysis should also consider relative benefits of using other devices.

The clinical evaluation report and the risk management report for the device should be based upon clinical evidence of the device for the intended use—including adverse events. For new devices that are evaluated based upon literature review of equivalent devices, Notified Bodies expect a Post-Market Clinical Follow-up (PMCF) study to be conducted in order to verify that the actual risk-benefit of the device is consistent with the conclusions of the clinical evaluation. In order to perform this analysis, a clinical expert is necessary to properly evaluate the risk-benefit ratio of the device, and to create a protocol for a PMCF study.

MEDDEV 2.12/2 rev 2, Post Market Clinical Follow-up Studies, indicates that the PMCF study protocol should indicate the study endpoints and the statistical considerations. In order to do this, your company will need to establish quantitative criteria for acceptability of the identified risks. Therefore, the existing 14971 Standard needs to be modified to clarify that risk acceptability criteria should be based upon clinical data, and evaluation of risks should be conducted at a later point in the risk management process (e.g., – as part of the overall risk-benefit analysis).

Impact of Deviation #2

As your company becomes aware of the second deviation between the 14971 Standard and the Essential Requirements of the device directives, your risk management team will need to change the risk management process to clarify when risk acceptability should be evaluated, and the risk management policy should specify how acceptability should be determined.

The risk management process at your company will need to specify that implementation of risk controls is required for all risks—regardless of acceptability. You should also consider eliminating the evaluation of risk prior to implementation of risk controls. Instead, your company should base acceptability of risk solely upon the clinical risk-benefit analysis, and should involve the manufacturer’s medical officer in making this determination.

Finally, your risk management process should specify the need for PMCF studies in order to verify that actual clinical data supports the conclusion that the risk-benefit ratio is acceptable over the lifetime of the device.

If you are interested in ISO 14971 training, we are conducting a risk management training webinar on October 19, 2018.

Posted in: Risk Management

Leave a Comment (6) →

Benefits of Incorporating Risk Management into Procedure Documents

By Guest Blogger, Brigid Glass
8971385878 db2fe2e49a q Benefits of Incorporating Risk Management into Procedure DocumentsThe author discusses benefits of incorporating risk management into procedure documents. An example procedure for Record Control is included.

When I was first introduced to FMEA many years ago, I loved it. I loved the systematic approach, and  particularly appreciated using a Process FMEA to explain to those involved with a production process why certain controls had been put in place. I enthusiastically taught FMEA to our engineers. At the time, our bubbly, buoyant, outcomes-focused Training Manager said to me, “You Quality people have such a negative outlook. You’re always looking for what can go wrong!”  Well yes, but it’s our role to prevent things from going wrong!  I’d found a tool to help me with that.

Next, there was EN 1441, a risk analysis standard that never satisfied, and always felt incomplete. ISO 14971 followed, covering the entire lifecycle of a product, with closed feedback loops.  So now, risks in product and process design were well covered, but ISO 13485 section 7.1 asks us to “establish documented requirements for risk management throughout product realization.”  Many of us would acknowledge that we could do better, even though we pass audits.  And what about the rest of the quality management system?  I know that when we document a procedure, we already apply risk management principles in our heads, but we usually don’t apply them systematically, or write down the results.

The Idea

Recently, Rob Packard and I started work on a project that requires us to generate a full set of documentation for a QMS, compliant with both U.S. and EU requirements, including ISO 13485 and ISO 14971. We each had our own ideas on how best to write a procedure, but this project provided us an opportunity to get some synergy going. Rob wanted to address risk management in each procedure. “Yes!” I said, thinking that here was a chance to fill that gap. But then it was my job to develop the template for the procedures and work out how to accomplish this…

My first results looked very complicated, so I took the KISS (Keep It Simple, Stupid) approach: one column for the hazards and consequences, and one for the risk control measures.

What I didn’t include:

  • I started with more complex hazard documentation (hazard ID, impact, trigger event, etc), but felt the benefits in the context of a procedure document were not balanced by the extra complexity and work required for analysis and training. It would be a hard sell to users within an organisation who were not used to the risk management approach.
  • I decided not to assess risks and controls quantitatively for the same reasons as in the bullet point above.
  • Initially, I included references to implementation, but this would be difficult to maintain as other documents changed.
  • I thought about verification of implementation of risk controls, then decided to leave that verification to reviewers.

Below is an example from a procedure for Record Control where records are completed on paper, then scanned as a pdf. My list won’t be the same as your list, but it is illustrative.

brigid chart 1 Benefits of Incorporating Risk Management into Procedure Documents

Standards and regulations are essentially a set of risk controls, so they are the first starting point when identifying hazards. The list should include direct risks to product, risks to the integrity of the QMS and regulatory risks. For those of us who have been in this industry for awhile, experience, past mistakes, questions fielded in external audits and observations of other systems will yield further hazards and appropriate controls. Audits provide the opportunity to update and refine the list and test the control measures.

Benefits of Incorporating Risk Management into Procedure Documents

  • Impresses your ISO 13485 auditor!
  • When first writing procedure documents, starting the writing process by reviewing the external requirements, and systematically writing the risk section, sharpens the mind as to what must be included in the procedure. This is the same approach as in design controls, where we include risk mitigators that apply to product design in the design inputs. This is part of planning in the PDCA cycle.
  • Supports future decision-making, in the same way that the risk file for a product is considered when a design is changed. The risk control section of a procedure provides the criteria against which any improvement or change can be assessed. Will it enhance the risk controls, or might it introduce a new hazard?
  • Serves as the basis for training on the procedure. Making visible the link between potential hazards and procedural controls much more convincing than saying, “Do this because the procedure says so,” or, “It’s in the procedure because the regs say so.”

This is part 1 in a series of blogs that leads up to our Roadmap to Iso 13485 Certification Courses

 

Posted in: Risk Management

Leave a Comment (0) →

Negligible Risks – Deviation #1 in ISO 14971

%name Negligible Risks   Deviation #1 in ISO 14971This blog reviews the treatment of the negligible risks, which is deviation #1 within the EN ISO 14971:2012 European normative risk management standard.

In 2012, the European National (EN) version of the Medical Device Risk Management Standard was revised, but there was no change to the content of Clauses 1 through 9. Instead, the European Commission identified seven content deviations between the 14971 Standard and the requirements of three device directives for Europe. This seven-part blog series reviews each of these changes individually.

Treatment of Negligible Risks in ISO 14971

The first deviation is specific to the treatment of negligible risks. In Annex D8.2, the ISO 14971 Standard indicates that the manufacturer may discard negligible risks. However, Essential Requirements in the three device directives require that “All risks, regardless of their dimension, need to be reduced as much as possible and need to be balanced, together with all other risks, against the benefit of the device.”

Common Misinterpretations

One of the most common mistakes is to confuse the concepts of a hazard, harm, and risk. Each of these terms is defined in the ISO 14971 Standard in section 2, but the common mistake is to think that the European Commission is saying that 100% of the hazards you identify need to be reduced as much as possible.

The intent is to require manufacturers to reduce risks, rather than hazards. The first step of the risk analysis process involves identifying hazards, but some of these hazards may never result in harm, due to risk controls that are inherent to the design your company has chosen. In addition, the severity of harm that a hazard may present could be so low that it may present no risk to the user or patient.

The best practice in risk management is to identify as many hazards as possible at the beginning of the risk analysis process, but then these hazards must be sorted into those hazards that will be analyzed for risk. One of the common phrases used in training is: “It is better to estimate the risk of 10% of 1,000 hazards than it is to estimate 50% of 100 hazards.”

If you follow the logic behind the phrase above, your team will need to estimate risk for 100 hazards, rather than 50 hazards. Your risk analysis team will also need to document the rationale behind categorization of hazards.

Categorizing Hazards

If a hazard is associated with adverse events in the Manufacturer and User Facility Device Experience (MAUDE) database for your device or a similar device, then you need to ensure that the risk associated with that hazard is assessed and there are adequate risk controls. This is also true for any hazard associated with a customer complaint that your company anticipates. Any hazard that presents a high potential severity of harm should also be included in your risk analysis. However, if a hazard is completely eliminated by the design of your device, then you do not need to include it in the risk analysis.

I recommend writing a hazard identification report that includes all the hazards that were identified. This report should also categorize the hazard. You only need two categories: 1) hazards to be analyzed for risk, and 2) hazards that do not require risk analysis. You need a rationale for each risk that you do not perform risk analysis for, and you need traceability to risk controls and the risk-benefit analysis for each hazard that you do analyze.

Example of a Rationale for Not Analyzing the Risk of a Hazard:

About 8 years ago, the United States Food and Drug Administration (USFDA) issued an alert cautioning physicians to avoid the use of hemostatic agents near the spinal column, due to the potential hazard of paralysis caused by the swelling of a hemostatic agent as it absorbs blood. My employer, Z-Medica, quickly received many customer inquiries asking about the safety of QuikClot near the spinal column. I was able to quickly respond that there was zero risk of QuikClot causing paralysis, because that particular hemostatic agent did not swell. Instead of absorption, the product adsorbed blood and did not change in size or shape during the adsorption process.

Impact of this Deviation

As companies become aware of this deviation between the 14971 Standard and the Essential Requirements of the device directives, I believe teams that are working on risk analysis and people that are performing a gap analysis of their procedures will need to be more careful about which hazards are identified in their risk management reports. The burden of showing traceability from hazards to risk controls and risk-benefit analysis is substantial. Therefore, it is important to be systematic about how hazards are identified, and to provide a clear justification for any hazards that are not included in the risk analysis.

The common phrase that has been used in risk management training classes should be reconsidered in light of feedback from the European Commission. Maybe a better phrase would be: “It is better to estimate the risk of 10% of 200 hazards than it is to estimate 50% of 20 hazards. However, it is important to provide a clear justification for any hazards that are not included in the risk analysis.”

If you are interested in ISO 14971 training, we are conducting a risk management training webinar on October 19, 2018.

 

Posted in: Risk Management

Leave a Comment (4) →

ISO Audit 14971:2012-4 Steps to Determining Compliance

The author provides 4 key steps to determining compliance when conducting an ISO audit of 14971:2012. Many auditing-related questions are also included.

Let’s say that you went ahead and purchased ISO 14971:2012, read Annex ZA, and identified a couple of gaps in your procedure. After you revised your Risk Management procedure to be compliant with the revised Standard, then what are you supposed to do?

Most QA Managers struggle over whether they should purchase ISO 14971:2012. I wrote a couple of blog postings about this matter, but my point was not to debate this question, but to ensure companies are aware that they need to be compliant with the MDD and the ISO 14971 Standard. The “changes” from the 2009 to the 2012 version are simply the European Commission reminding manufacturers that there are seven aspects of the ISO 14791 Standard that  do not meet the requirements of the MDD. Therefore, if your company has already verified that your risk management process is compliant with the MDD–then you have nothing to change. However, if your risk management process is only compliant with ISO 14971:2009, then you need to revise your processes and procedures to address these seven aspects. 

4 Steps in Conducting an ISO Audit 14971:2012

Once you have made your revisions, how do you audit for compliance with ISO 14971:2012?

Step 1: Planning the Audit

This will be an internal audit, and since you (the QA Manager) are the process owner for the risk management process, you personally cannot audit this process. You need to assign someone that has the technical skill to perform the audit, but this person cannot be the process owner (you) or a direct report to the process owner (the rest of the QA department). Fortunately, the Director of Engineering is also trained as an internal auditor at your company. She is trained on ISO 14971:2009, but she is not trained on ISO 14971:2012. To address this gap, she must read the updated Standard to understand what’s new.

%name ISO Audit 14971:2012 4 Steps to Determining Compliance

Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

She has participated in risk management activities, but each product development engineer participates in risk management activities for their own design projects. Therefore, she has several projects she can sample risk management records from without auditing her own work. You have communicated that you need this audit finished sometime in December, because you want any CAPA’s resulting from the audit to be finalized before the next Management Review at the end of January. The timing of the Management Review is important, because the risk management procedure requires that top management assess the effectiveness of the risk management process during Management Review meetings.

There are no previous audit findings to close from the last audit of the risk management process, but the Director of Engineering has seven specific items to emphasize from the 2012 revision of the Standard, and a revised procedure for risk management. Therefore, she will prepare for the audit by identifying some new interview questions to specifically address these changes–as well as some more generic, open-ended questions.

Specific Questions for 7 Items in ISO 14971:2012, Annex ZA:

1. How does the risk analysis evaluate the acceptability of risks in the lowest category? (This is a leading question, but it is specifically designed to determine if negligible risks are discarded).

2. Please provide a few examples of how risks in the lowest category were reduced. (Sections 1 and 2 of the Annex I require all risks to be reduced as far as possible, and for all risks to be evaluated for acceptability. The wording of this question also allows auditors flexibility in their sampling).

3.  How did the design team determine when they had implemented sufficient risk controls to minimize risks? (Many companies use a color-coded matrix as a quasi-objective method for determining when risks are adequately reduced. This process is often referred to as the ALARP concept. Annex ZA specifically prohibits using economic considerations as part of this determination).

4. How did you conduct a risk-benefit analysis? (The Standard allows for performing a risk-benefit analysis when overall residual risks exceed the acceptability criteria as outlined in the risk management plan. However, the MDD requires an overall risk-benefit analysis in Section 1 of Annex I. Section 6 also requires that a risk-benefit analysis be performed for each individual risk).

5. How were risk control options selected? (Section 2 of the MDD implies that the manufacturer shall review All the control options and pick the most appropriate ones. Therefore, the auditor should specifically look for evidence that the team systematically reviewed all possible control options to reduce risks–rather than stopping as soon as the risks were reduced to an acceptable level).

6. What were your team’s priorities for implementation of risk control options? (It’s possible that the previous question will be sufficient to gather evidence that risk controls were implemented with the required prioritization, as specified in the MDD. However, this question would be used as a follow-up question if it is not clear that the team prioritized the risk control options in accordance with Section 2 of Annex I).

7. How was effect of labeling and warnings in the instructions for use incorporated into the estimation of residual risks? (Almost every company remembers to include residual risks in their IFU as a warning or caution statement. However, Section 2 of Annex I does not allow for including this information given to the users as a method of reducing risks. Therefore, in a Design FMEA, you would not list labeling and IFUs in your column for current risk controls when you determine the risk. This should be identified as an action to be taken–with no impact on the score for residual risk).

%name ISO Audit 14971:2012 4 Steps to Determining ComplianceThe above questions are not examples of using the process approach, but each question is phrased in an open-ended manner to maximize the objective evidence gathered during the interview process. If you are doing a process audit, it’s still acceptable to include questions that use the element approach.

Generic Questions:

1. When was the ISO 14971:2012 version of the Standard added to the controlled list of external Standards?

2. Please provide examples of where you have updated the Essential Requirements Checklist (a Technical File document) to reference the newest revision of ISO 14971:2012, and please show at least one example of how the risk management report was updated to reflect this revision.

3. How did you verify training effectiveness for the design team specific to the updated risk management procedure prior to conducting a risk analysis?

%name ISO Audit 14971:2012 4 Steps to Determining ComplianceThese generic questions do not require reading the ISO 14971:2012 Standard. Instead, each question forces the auditee to demonstrate their knowledge of the revised Standard by answering open-ended interview questions. Each of these questions is also designed to test linkages with other support processes. This is an example of how to use the process approach.

Step 2: Conducting the Audit

The next step of the auditing process is to conduct the audit. During the audit, the Director of Engineering will gather objective evidence of both conformity and nonconformity for the risk management process. The generic interview questions that were developed allow her to evaluate the effectiveness of linkages between the risk management process and other processes, such as:

1) Document control

2) Creating technical documentation for regulatory submissions

3) The training process

Specific questions verify that each of the seven elements identified in Annex ZA of ISO 14971:2012 are adequately addressed in the revised procedure. When the audit is completed, the auditor will have a closing meeting with the process owner (you) and the auditee(s), so that everyone is clear what the findings were, and if there were any nonconformities. This is the time to clarify what needs to be done in order to prevent each nonconformity from recurring.

Step 3: Writing the Report & Taking Corrective Action(s)

This is no different from any other audit, but it is critical to have the report completed soon enough so that CAPA’s can be initiated (not necessarily completed) prior to the Management Review.

Step 4: Verifying Effectiveness of Corrective Action(s)

Many people struggle with verifying effectiveness of corrective actions–regardless of the process. My advice is to identify a process metric to measure the effectiveness. Then the effectiveness check is objective. For example, monitoring the frequently of updates to the list of external standards can help verify that the process for monitoring when Standards are updated is effective. Likewise, the frequency of updates to the Essential Requirements Checklist and the risk management records referenced in the Essential Requirements Checklist indicates if the risk management process is  being maintained. Finally, monitoring the lag between the time procedures are updated and when the associated training records are updated quickly identifies if there is a systemic problem with training or if a training gap is just an example of a single lapse.

Posted in: Risk Management

Leave a Comment (7) →

Do I Need to Purchase the EN 14971:2012 version?

The author explains why it is not necessary to purchase the EN 14971: 2012 version, and provides an option to obtain information you may need.

What?!

I thought the current version was 2007?

No, there was a minor correction in 2009, but I haven’t heard about a 2012 version?

Great, now I have to buy another $300 Standard that will tell us nothing new.

STOP!

If the above conversation sounds familiar, hopefully, this blog will help.

Question 1: What is the current version?

Answer 1: EN 14971 was revised to 2012 on July 6, 2012. The previous 2009 version was withdrawn. The ISO version is not changing–just the EN version.

Question 2: What’s new in 2012?

Answer 2: Only the three Annexes related to harmonization with the three directives (MDD, AIMDD and IVDD) were updated. The content of the Standard itself has not changed.

Question 3: Do I need to buy this latest revision of the Standard…which really hasn’t changed since 2007?

Answer 3:  No…unless you still have the 2000 version. (just my personal opinion…not anyone else necessarily agrees)

And Here’s Why…

Historically, Annex ZA was the annex at the back of a Standard that would explain how it is harmonized with the European Directives. However, in 2009, Annex ZA was separated into ZA, ZB and ZC. Each of these Annexes explained how the current version of ISO 14971  (then ISO 14971:2007) differs from each of the three directives. In addition, there was a correction to Figure 1 (i.e., – arrow in the wrong location). NevilleClarke provided a good summary of these minor changes that occurred in 2009. The European Commission was concerned with some of the differences between the 2009 Standard and the Directives. Therefore, the Standard has been updated to clarify these differences.

There are seven technical deviations from the Standard that are required for compliance with the European Directives. Marcelo Antunes is an expert on Standards, and he accurately describes these deviations as “weird” in a discussion thread on Elsmar Cove’s Forum. The deviation that seems to have caught the most attention is the requirement to reduce ALL risk to “as low as possible” (ALAP) rather than to a level that to “as low as reasonably practicable” (ALARP concept). The “ALAP” acronym was a joke, but it wouldn’t be the first time that something like this stuck (i.e., – SWAG).

EN 14971: 2012 Version: An Alternative Approach

If you sleep with a label maker under your pillow, you should buy the new BS EN 14971:2012 version,  so you can ensure that you are staying in compliance with each of these 7 deviations, and that you have considered the implications fully in your procedure for Risk Management. However, if you are a practical person that prefers not to upset the entire development team, I recommend a different approach.

1. Download a copy of the relevant Directive from the Europa Website

2. Using Adobe, search the entire Directive for the word “risk”:

AIMDD = 24 times

MDD = 55 times

IVDD = 34 times

3. Systematically review where the word “risk” is used to determine if you need to make adjustments for your CE Marked products. If you already have a CE Mark, there should be no changes required to your risk management documents. Your procedures might need clarification to observe the requirements of the Directive when there is a difference between the Standard and the Directive.

Last Question: What is your Notified Body auditor going to do?

Final Answer: I’m not sure, because every auditor is a little different in their approach. However, as an instructor, I would teach an auditor to ask open ended questions, such as: “How did you determine if there is an impact upon your procedures and design documentation with regard to the updated Standard?” (i.e., – impact analysis). If the company provides an impact analysis and explains why the existing risk documentation and procedure should not change, I believe this meets the requirements for “equivalency with the State of the Art.”

Honestly, I haven’t seen one single company that was 100% in compliance with the “letter” of the Directives or the Standard. Sometimes, rationale thought must overcome political compromises and irrational behaviors.

On the other hand, it’s always possible that these 7 deviations, and the information on corrective action, will fundamentally change the way your company approaches risk management (I just dare you to bring it up at your next management review).

If you would like a second opinion, the Document Center’s Standard Forum says, “As you can see, this material is essential to conformance with the EN requirements and will make the purchase of the EN edition (BS EN ISO 14971 is the official English language edition) mandatory for medical device manufacturers certifying to the standard for sales in Europe.” FYI…Document Center’s Standard Forum sells Standards. You can buy this one from them for $324.

 

Posted in: Risk Management

Leave a Comment (11) →

Learning Pyramid – 4 Levels of Learning

The author discusses the four levels of learning in the Learning Pyramid, and the lessons learned when he taught an ISO 14971 Risk Management course.%name Learning Pyramid   4 Levels of Learning

 

I am in Canada, it’s almost midnight, and my client has me thinking so hard that I can’t sleep. I am here to teach the company’s Canadian facility about ISO 14971:2007—the ISO Standard for Risk Management of medical devices.

Most of the companies that request this training are doing so for one of two reasons: 1) several of their design engineers know almost nothing about risk management, or 2) they have several design engineers that are quite knowledgeable with regard to risk management, but these engineers have not maintained their credentials, and their last risk management training was related to the 2000 version of the Standard. This company falls into the second category.

I always tell students that I learn something by teaching each course. From this company, however, I have learned so much. This company has forced me to re-read the Standard a number of times and reflect on the nuances of almost every single phrase. I have learned more about this Standard in one month than I learned in the 3.5 years since I first took the course I am now teaching. 

The four levels of the Learning Pyramid

I have developed a model for learning that explains this phenomenon. I call this model the “Learning Pyramid.” At the base of the pyramid, there are “Newbies.”

This is the first of four levels. At the base, students read policies and procedures with the hope of understanding.

In the second level of the pyramid, the student is now asked to watch someone else demonstrate proper procedures. One of my former colleagues has a saying that explains the purpose of this process well, “A picture tells a thousand words, but a demonstration is like a thousand pictures.” This is what our children call “sharing time,” but everyone over 40 remembers this as “show and tell.”

In the third level of the pyramid, the student is now asked to perform the tasks they are learning. This is described as “doing,” but in my auditing courses, I refer to this process as “shadowing.” Trainees will first read the procedures for Internal Auditing (level 1). Next, trainees will shadow the trainer during an audit as a demonstration of proper technique (level 2). During subsequent audits, the trainees will audit and the trainer will shadow the trainee (level 3). During this “doing” phase, the trainer must watch, listen and wait for what I call the “Teachable Moment.” This is a moment when the trainee makes a mistake, and you can use this mistake as an opportunity to demonstrate a difficult subject.

Finally, in the fourth level of the Learning Pyramid, we now allow the trainee to become a trainer. This is where I am at—so I thought. I am an instructor, but I am still learning. I am learning what I don’t know.

Teaching forces you back to the bottom of the Learning Pyramid

The next step in the learning process is to return to the first level. I am re-reading the Standard and procedures until I really understand the nuances that I was unaware of. Then, I will search for examples in the real world that demonstrate these complex concepts I am learning. After searching for examples, I will test my knowledge by attempting to apply the newly acquired knowledge to a 510(k) or CE Marking project for a medical device client. Finally, I will be prepared to teach again.

This reiterative process reminds me of the game Chutes and Ladders, but one key difference is that we never really reach the level of “Guru.” We continue to improve, but never reach our goal of perfection…For further inspiration, try reading “Toyota Under Fire.”

Posted in: Education, Risk Management

Leave a Comment (5) →

Contract Manufacturers Need Strong Risk Management Processes

This blog discusses why contract manufacturers need to have strong risk management process and your company needs to help your contract manufacturers.

Risk management is not our responsibility Contract Manufacturers Need Strong Risk Management Processes

Can contract manufacturers exclude risk management from the scope of their quality system?

Most contract manufacturers in the medical device industry exclude design from their Quality Management Systems. Unfortunately, most of the contract manufacturers also associate risk management with only the design process. Risk Management cannot be “not applicable” in an ISO 13485 Quality Management System. The requirement of section 7.1 is: “The organization shall establish documented requirements for risk management throughout product realization. Records arising from risk management shall be maintained.” The Standard also references ISO 14971 as a source of guidance on Risk Management.

Have you experienced an audit dialogue at a contract manufacturer similar to this?

The auditor asks, “How do you manage risk throughout the production process?” Then the auditee responds, “That is the responsibility of our customers. We will prepare a risk analysis if customers pay for it, but usually customers do the risk analysis.”

For a contract manufacturer, compliance with ISO 14971 is not my primary concern as an auditor. My primary concern is to verify that contract manufacturers analyze risks associated with the processes that they perform, and do their best to minimize those risks. What I don’t understand is why more companies don’t want to have strong risk management processes. Risk management is how we prevent bad things from happening. Bad things like scrap, complaints and recalls. Should we expect our suppliers to have a strong risk management process?

Duh.

Why your company needs to be involved in the risk management process?

Contract manufacturers should be doing everything they can to get better at risk management. During pre-production planning they should be asking, “What happens if…” The contract manufacturer knows best HOW things will fail in production, while the customer knows best WHAT happens when things fail in production. In order to be safe and effective, both companies need to collaborate on risk analysis.

In any risk analysis, you need to estimate the severity of potential harm and the probability of occurrence of that harm. For production defects, the contract manufacturer can estimate the probability of occurrence of defects (i.e., P1 in Annex E of ISO 14971:2007), but the probability of occurrence of harm is less. The probability of occurrence of harm is the product of multiplying P1 and P2. The probability that occurrence will result in harm is P2, and P2 is a number that is less than 100% or 1. Your company can gather pre-market clinical data and post-market clinical data to estimate P2, but prior to launching your product you can only guess at the value of P2. Your contract manufacturer, however, is not able to estimate P2 at all. It’s ok estimate risk without P2 during the design phase, because this will overestimate risks and result in more conservative decisions.

In addition to P2, your contract manufacturer is also not capable of estimating the severity of potential harm. As the designer of the medical device, you will know best how your device is used and what the likely clinical outcomes are when a device malfunctions. There may even be multiple possible clinical outcomes. The contract manufacturer knows what can go wrong during manufacturing, but you will need to define the clinical outcomes due to malfunctions.  

Why do contract manufacturers avoid doing risk analysis?

The reason contract manufacturers avoid doing risk analysis is because it’s time consuming and tedious.

Too bad, so sad.

Balancing my checkbook is time consuming and tedious too, but I balance my checkbook to prevent an overdraft charge. Not doing risk analysis can be much more painful. Scrapping out a part can cost tens or hundreds of dollars. Complaints can cost thousands of dollars. Recalls can cost millions of dollars.

If I owned a contract manufacturing company, I would ensure that everyone in the company is involved in risk management. We don’t want scrap, we can’t afford mistakes that lead to complaints and a recall could put us out of business.

Posted in: Risk Management

Leave a Comment (1) →
Page 2 of 2 12