Author name: Robert Packard

Negligible Risks – Deviation #1 in ISO 14971

This blog reviews the treatment of the negligible risks, which is deviation #1 within the EN ISO 14971:2012 European normative risk management standard.

%name Negligible Risks   Deviation #1 in ISO 14971

In 2012, the European National (EN) version of the Medical Device Risk Management Standard was revised, but there was no change to the content of Clauses 1 through 9. Instead, the European Commission identified seven content deviations between the 14971 Standard and the requirements of three device directives for Europe. This seven-part blog series reviews each of these changes individually.

Treatment of Negligible Risks in ISO 14971

The first deviation is specific to the treatment of negligible risks. In Annex D8.2, the ISO 14971 Standard indicates that the manufacturer may discard negligible risks. However, Essential Requirements in the three device directives require that “All risks, regardless of their dimension, need to be reduced as much as possible and need to be balanced, together with all other risks, against the benefit of the device.”

Common Misinterpretations

One of the most common mistakes is to confuse the concepts of hazard, harm, and risk. Each of these terms is defined in the ISO 14971 Standard in section 2, but the common mistake is to think that the European Commission is saying that 100% of the hazards you identify need to be reduced as much as possible.

The intent is to require manufacturers to reduce risks, rather than hazards. The first step of the risk analysis process involves identifying hazards. Still, some of these hazards may never result in harm, due to risk controls that are inherent to the design your company has chosen. Also, the severity of harm that a hazard may present could be so low that it may present no risk to the user or patient.

The best practice in risk management is to identify as many hazards as possible at the beginning of the risk analysis process. Still, then these hazards must be sorted into those hazards that will be analyzed for risk. One of the common phrases used in training is: “It is better to estimate the risk of 10% of 1,000 hazards than it is to estimate 50% of 100 hazards.”

If you follow the logic behind the phrase above, your team will need to estimate risk for 100 hazards, rather than 50 hazards. Your risk analysis team will also need to document the rationale behind the categorization of hazards.

Categorizing Hazards

If a hazard is associated with adverse events in the Manufacturer and User Facility Device Experience (MAUDE) database for your device or a similar device, then you need to ensure that the risk associated with that hazard is assessed and there are adequate risk controls. This is also true for any hazard associated with a customer complaint that your company anticipates. Any hazard that presents a high potential severity of harm should also be included in your risk analysis. However, if a hazard is entirely eliminated by the design of your device, then you do not need to include it in the risk analysis.

I recommend writing a hazard identification report that includes all the hazards that were identified. This report should also categorize the hazard. You only need two categories: 1) hazards to be analyzed for risk, and 2) hazards that do not require risk analysis. You need a rationale for each risk that you do not perform risk analysis for, and you need traceability to risk controls and the risk-benefit analysis for each hazard that you do analyze.

Example of a Rationale for Not Analyzing the Risk of a Hazard

About eight years ago, the United States Food and Drug Administration (USFDA) issued an alert cautioning physicians to avoid the use of hemostatic agents near the spinal column, due to the potential hazard of paralysis caused by the swelling of a hemostatic agent as it absorbs the blood. My employer, Z-Medica, quickly received many customer inquiries asking about the safety of QuikClot near the spinal column. I was able to quickly respond that there were zero risks of QuikClot causing paralysis because that particular hemostatic agent did not swell. Instead of absorption, the product adsorbed blood and did not change in size or shape during the adsorption process.

Impact of Deviation #1 about “Negligible Risk”

As companies become aware of this deviation between the 14971 Standard and the Essential Requirements of the device directives, I believe teams that are working on risk analysis and people that are performing a gap analysis of their procedures will need to be more careful about which hazards are identified in their risk management reports. The burden of showing traceability from hazards to risk controls and risk-benefit analysis is substantial. Therefore, it is important to be systematic about how hazards are identified and to provide a clear justification for any hazards that are not included in the risk analysis.

The common phrase that has been used in risk management training classes should be reconsidered in light of feedback from the European Commission. Maybe a better phrase would be: “It is better to estimate the risk of 10% of 200 hazards than it is to estimate 50% of 20 hazards. However, it is important to provide a clear justification for any hazards that are not included in the risk analysis.”

If you are interested in ISO 14971 training, we are conducting a risk management training webinar on October 19, 2018.

Negligible Risks – Deviation #1 in ISO 14971 Read More »

An Auditor’s Best Practices in Issuing a Major Nonconformity

%name An Auditors Best Practices in Issuing a Major Nonconformity

From the opening meeting through the audit and closing meeting, the author describes an auditor’s best practices in issuing a major nonconformity.

As an auditor, one of the most important (and difficult) things to learn is how to issue a nonconformity—especially a major. This is usually done at the closing meeting of an audit, but the closing meeting is not where the process of issuing the nonconformity begins. Issuing a nonconformity starts in the opening meeting.

ISO 19011:2011 is the official guidance document for auditors of Quality Management Systems, and ISO 13485 is the quality system standard for medical device manufacturers. Section 6.4.2 of this Standard explains best practices for an opening meeting. The last five items in this section are critical to preparing the client for potential nonconformities:

  1. Method of reporting audit findings, including grading, if any
  2. Conditions under which the audit may be terminated
  3. Time and place of the closing meeting
  4. How to deal with possible findings during the audit
  5. System for feedback from the auditee on findings or conclusions of the audit
  6. Process for complaints and appeals
Methods of Reporting and Grading Nonconformities

The auditor should be crystal clear in their description of minor and major nonconformities or any other grading that will be used. The auditor should also make it clear that they are looking for conformity rather than nonconformity. This is an audit—not an inspection. Typically, a minor nonconformity is described as “a single lapse in the fulfillment of a requirement,” while a major nonconformity is described as one of the following: 1) “a total absence in the fulfillment of a requirement,” 2) “repetition of a previous nonconformity,” 3) “failure to address a previously identified minor nonconformity,” or 4) “shipment of non-conforming product.” When the auditor is in doubt, then the finding is minor, and never a major. For a major nonconformity to be issued, there can be no doubt.

Conditions for Termination

The option to terminate an audit is typically reserved for a certification audit where a major nonconformity is identified, and there is no point in continuing. Termination is highly discouraged, because it is better to know about all minor and major nonconformities right away, instead of waiting until the certification audit is rescheduled. The certification body will charge you for their time anyway.

Another reason for termination is when an auditor is unreasonable or inappropriate. This is rare, but it happens. If the audit is terminated, you should communicate this to upper management at the certification body and the company—regardless of which side of the table you sit. For FDA inspections, this is not an option. For audits performed by Notified Bodies, there is the possibility of suspension of a certificate in response to audit termination. Therefore, I always recommend appealing after the fact, instead of termination. Appealing also works for FDA inspections.

How to Deal with Findings

All guides and auditees should be made aware of possible findings at the time an issue is discovered. This is important so that an auditee has the opportunity to clarify the evidence being presented. Often, nonconformities are the result of miscommunication between the auditor and the auditee. This frequently happens when the auditor has a poor understanding of the process being audited. It is a tremendous waste of time for both sides when this occurs. If there is an actual nonconformity, it is also important to gather as much objective evidence as possible for the auditor to write a thorough finding and for the auditee to prepare an appropriate corrective action plan in response to the discovery.

%name An Auditors Best Practices in Issuing a Major Nonconformity
Feedback from the Auditee

As an auditor, I always encourage auditees to provide honest feedback to me directly and to management, so that I could continue to improve. If you are giving feedback about an internal auditor or a supplier auditor, you should always give feedback directly before going to the person’s superior. You are both likely to work together in the future, and you should give the person every opportunity to hear the feedback first-hand.

When providing feedback from a third-party certification audit, you should know that there will be no negative repercussions against your company if you complain directly to the certification body. At most, the certification body will assign a new auditor for future audits and investigate the need for taking action against the auditor. In all likelihood, any action taken will be “retraining.” I never fired somebody for a single incident—unless they broke the law or did something unsafe. The key to providing feedback, however, is to be objective. Give specific examples in your complaint, and avoid personal feelings and opinions.

Complaints and Appeals

As the auditee, you should ask for the contact information of the certification body during the opening meeting. Ask with a smile—just in case you disagree, and so you can provide feedback (which might be positive). As the auditor, you should always make contact information for the certification body available. If you are conducting a supplier audit or an internal audit, you probably know the auditor’s boss, and there is perhaps no formal complaint or appeals process. In the case of a supplier audit, the customer is always right—even when they are wrong.

During the Audit

During the audit, you should always make the guide(s) and process owner(s) aware of any potential nonconformities as you find them. This is their opportunity to clarify the objective evidence for you and to explain why there is not a nonconformity. Often, at this point in the audit, I will refer to the Standard. I will identify the specific requirement(s) and show the process owner. I will say, “This is what I am trying to verify. Do you have anything that would help address this requirement?” If the process owner is unsure of how to meet the requirement, often, I will provide an example of how this requirement is addressed in other areas or at other companies.

If the audit is a multi-day audit, I will review the potential nonconformities at the end of the day and allow the auditee to provide additional objective evidence in the morning. If it is the last day of the audit, or it is a single-day audit, I will give auditees until the closing meeting to provide the objective evidence. Often, I will use this opportunity to explain what would be considered a minor nonconformity and what would be a major nonconformity. Usually, I can say, “This is not a major nonconformity because…”

%name An Auditors Best Practices in Issuing a Major Nonconformity

Closing Meeting

The closing meeting should be conducted as scheduled, and the time/location should be communicated to upper management in the audit agenda and during the opening meeting. Top management won’t be happy about nonconformities, but failure to communicate when the closing meeting will be conducted will irritate them further.

At the closing meeting, the auditee should never be surprised. If an issue remains unfulfilled at the closing meeting, the auditee should be expecting a minor nonconformity—unless the issue warrants a major nonconformity. Since a minor nonconformity is described as “a single lapse in the fulfillment of a requirement,” it is difficult for an auditee to argue that an issue does not warrant a minor nonconformity. Typically, the argument is that you are not consistent with other auditors. The most common response to that issue is, “Audits are just a sample, and previous auditors may not have seen the same objective evidence.” The more likely scenario, however, is that the previous auditor interprets requirements, instead of reviewing requirements with the client, and ensuring both parties agree before a finding is issued.

If a finding is major, the auditee should have very few questions. Also, I often find the reason for a major nonconformity is a lack of management commitment to address the root cause of a problem. Issuing a major nonconformity is sometimes necessary to get management’s attention.

Regardless of the grading, all audit findings will require a corrective action plan—even an FDA warning letter requires a CAPA plan. Therefore, a major nonconformity is not a disaster. You just need to create a more urgent plan for action.

An Auditor’s Best Practices in Issuing a Major Nonconformity Read More »

EU Medical Device Directive: 6 New Essential Requirements

%name EU Medical Device Directive: 6 New Essential RequirementsEssential Requirements (ER) changes in the proposed EU Medical Device Regulations versus the ER in Annex I of the EU Medical Device Directive are reviewed.

Click HERE if you want to receive future “European Regulatory Updates” by email. Just provide your name, company, phone number, and email address where you would like to receive updates.

Annex I of the European Medical Device Directive (http://bit.ly/M5MDD) is titled “Essential Requirements.” Most companies demonstrate that their device meets the 13 Essential Requirements (ERs) by creating an Essential Requirements Checklist (ERC). I have no idea what the origin of the ERC is, but you know that regulators love tables and checklists. This particular checklist is so commonly used that the Global Harmonization Task Force (GHTF) included an example of an ERC, called an “Essential Principles Checklist” (EPC) at the end of a guidance document on how to create Summary Technical Documentation (STED) for In Vitro Diagnostic devices (http://bit.ly/STEDIVD)—which is now maintained on the IMDRF.org website.

On September 26, 2012, the European Commission released a proposal for new EU Medical Device Regulations (http://bit.ly/EUProposal). This proposal still includes ERs in Annex I, but there are 19 ERs in the proposal. One regulatory professional recently sent me a follow-up question in response to an audio seminar I conducted in November (). Her question was, “What are the six new ERs?”

A few of the early reviews of the proposal indicated that there were no significant changes. Still, I have learned the hard way that you should always go to the source and verify the information for yourself (i.e., – Genchi Genbutsu). Here’s what I found:

General Requirements (ER 1-6a)

  1. No real change to this requirement.
  2. This requirement was reworded to clarify the intent (see Annex ZA of EN 14971:2012 for more info @ http://bit.ly/ISO14971-2012changes).
  3. It appears as though the Commission thought the current ER 3 was redundant, and the requirement was addressed by ER 1 and ER 5 already.
  4. This is now the new ER 3, and the requirement now clarifies how Notified Bodies shall apply this requirement in cases where a lifetime of the device is not stated.
  5. This is now the new ER 4, and there is no real change.
  6. This is now the new ER 5, and the wording has been clarified.

ER6a is conspicuously missing from the proposed ERs, but don’t get excited. Clinical evaluations are still required as part of the Technical Documentation in Annex II, Section 6.1c: “the report on the clinical evaluation in accordance with Article 49(5) and Part A of Annex XIII.”

Chemical, Physical & Biological Properties (ER 7)

ER 7.1 has one new requirement: “d) the choice of materials used, reflecting, where appropriate, matters such as hardness, wear and fatigue strength.” ER 7.2 and 7.3 remain unchanged. ER 7.4 has been simplified to what is proposed as the new, shorter ER 9. ER 7.5 is now the new ER 7.4, and the changes reflect the current status of phthalate regulations and similar issues. ER 7.6 is now the new ER 7.5, but there is no change to the content. The new ER 7.6 requires that manufacturers address risks associated with the size and properties of particles, especially nanomaterials. The changes related to this section will impact certain device types more than others—such as orthopedic implants.

Infection & Microbial Contamination (ER 8)

ER 8 is still ER 8, but ER 8.1 is now prescriptive regarding design solutions, and the current ER 8.2 is now the new ER 10. The new ER 10 is expanded and references the new EU regulations regarding devices manufactured utilizing tissues or cells of animal origin: Commission Regulation (EU) No 722/2012 of August 8, 2012 (http://bit.ly/AnimalTissueReg). The new ER 8.2 is a new requirement that was an oversight of the MDD, and the new ER 8.7 now clarifies that the labeling must differentiate sterile and non-sterile versions of the product; packaging is no longer an acceptable mechanism for differentiation. The balance of ER 8 remains unchanged.

Construction & Environmental Properties (ER 9)

This ER is now identified as the new ER 11, and this section is expanded. This reflects the emphasis on the need to evaluate the safety of devices with accessories, compatibility with other devices, and the effects of the use environment.

Devices with a Measuring Function (ER 10)

This ER is now identified as the new ER 12, but ER 10.2 from the current Directive appears to be missing. What’s up?

Take a look at the new ER 11. ER 10.2 is now the new ER 11.6.

Protection Against Radiation (ER 11)

This ER is now identified as the new ER 13, but there is nothing new.

Requirements for Devices Connected to or Equipped with an Energy Source (ER 12)

ER 12.1 and 12.1a are now ER 14. This section is specific to software requirements and has more detail than the current Directive. IEC 62304:2006, “Medical device software – Software life cycle processes,” is the Standard that will be expected by Notified Bodies as a reference for ER 14. ER 12.2 through ER 12.6 is now ER 15, but there is nothing new. Section ER 12.7 and its sub-parts are now addressed by ER 16. ER 12.8 and its subparts are now addressed by ER 17.

Information Supplied by the Manufacturer (ER 13)

This is now identified as ER 19: “Label and Instructions for Use.” This section is simplified from ER 13 (i.e., – there are fewer sections), but this ER does not seem to be any shorter. ER 19.1 has subparts a-g, and this ER section incorporates the concepts previously addressed by ER 13.1, 13.2, 13.4, and 13.5. ER 19.2 is a new and improved version of the previous ER 13.3 specific to labeling requirements. This labeling section is expanded from subparts “a” through “n” to “a” through “q.” The UDI requirement is subpart “h.” ER 13.6 is now ER 19.3 specific to the Instructions For Use (IFU). This section is expanded from subparts “a” through “q” to “a” through “t.”

The number of subparts to ER 19.3 doesn’t reflect the additional requirements for IFUs that are proposed by the Commission. The subsections of this part warrant special attention. Items that frequently are found missing from IFUs on the market today include:

  1. ER 19.3c – performance intended by the manufacturer
  2. ER 19.3h – installation and calibration instructions
  3. ER 19.3k – how to determine if a reusable device should be repaired/replaced
  4. ER 19.3m – restrictions on combinations with other devices
  5. ER 19.3o – detailed warning information
  6. ER 19.3p – information about safe disposal of the device
  7. ER 19.3t – notice to user/patient to report adverse events

ER 18 – Use by Lay Persons

This is a short section, but the requirement is new. There are no additional requirements for products intended for use by a layperson. The risk management report, design validation, and clinical evaluation report will need to include specific evidence to demonstrate conformity with this ER. The post-market surveillance plan for these products should carefully verify the accuracy of risk estimates. Post-Market Clinical Follow-up (PMCF) studies would be challenging in the past. Still, the prevalence of social media and product registration databases may facilitate conducting PMCF studies for these products in the future.

Australia & Canada

There is also an EPC that is required by the Therapeutic Goods Administration (TGA) in Australia, (http://bit.ly/EPCTGA) and  Therapeutics Product Directorate (TPD) in Canada (http://bit.ly/CanadianSTED). If you would like to learn more about the Essential Principles of Safety and Performance, you should also review the GHTF guidance document on this topic (http://bit.ly/EPSafetyPerf) on the IMDRF.org website. This 2012 version of the document supersedes GHTF/SG1/N041:2005.

I have observed the approval of products where the European ERC was submitted in place of an EPC for Australia and Canada. I guess they are a little more rational than some other regulators, but if you have experienced any “push back” regarding this approach, please share this by posting a comment or by sending an email.

If you need assistance with medical device CE Marking, or you are interested in training on CE Marking, please contact Medical Device Academy at rob@13485cert.com. Medical Device Academy is developing a webinar series specifically for this purpose. You can also call me by phone @ +1.802.258.1881. For other blogs on the topic of “CE Marking,” please view the following blog category page: http://robertpackard.wpengine.com/category/ce-marking/.

EU Medical Device Directive: 6 New Essential Requirements Read More »

A 6 Step Approach if You Disagree With a Notified Body Auditor

The author’s first certification audit experience is discussed, and we review six different approaches to take if you disagree with a notified body auditor.

My first certification audit ever didn’t go so well. The reason it didn’t go well is that the auditor wrote nonconformities that my boss and our regulatory consultant didn’t agree with. At the time, I was too inexperienced to know how to handle it. My boss and the consultant, however, totally lost it. I’ve never seen veins that big in someone’s forehead–even in cartoons.

I asked them both to leave the room because I was afraid to “push back” on the auditor. Many Management Representatives feel the same way that I did during that initial certification audit. The best way to summarize our concerns is with the following picture:

kodiak A 6 Step Approach if You Disagree With a Notified Body Auditor

Recently another LinkedIn group member emailed me to say that they have seen several auditors for registrars identifying nonconformities that represented their own personal opinions rather than specific requirements of the Standard. For example, there is a requirement to assign management responsibilities and document it, but there is no requirement to have an organization chart.

Another common mistake is when auditors insist that a company must create a turtle diagram for every single process. I support the use of turtle diagrams 100%, but the only requirement in the Standard is to use the process approach–not turtle diagrams specifically.

My favorite is my own personal mistake. I wrote a nonconformity for not having a process for implant registration cards for a company that was planning to ship a high-risk implant product to Canada. There is a requirement for implant registry cards, but I forgot that Canada defines “implants” in this case as only a very short list of implant devices–not implants in general.

Auditors are human. These are audit findings–not a jail sentence. Everyone needs to remember that the worst that can happen is that you receive a nonconformity. If the auditor finds a nonconformity, then you need to develop a CAPA plan. If the auditor finds nothing, you still need to do your own internal audits to identify nonconformities and continuously improve processes.

What Should You Do When an Auditor is Wrong?

I recommend that you “push back,” but you need to know-how. Many consultants suggest saying, “Can you show me in the Standard where it says I have to do that?” That’s just like poking a bear. If you do it once, it’s annoying. If you do it multiple times, an auditor might just eat you.

One Management Representative did that to me after I had taken the time to review the requirements with him. I responded by holding the ISO 13485 Standard in front of him and reciting clause 7.3.2. He responded by saying, “Well, that’s up for interpretation.” I offered to recite the ISO 14969 guidance document for him, but his boss told him to shut up.

This certainly wasn’t the only time a client pushed back during a registration audit, but other clients have had the sense to argue about things they understood.

One of the clients I audited said that he would change the topic to the auditor’s favorite sports team. That’s one approach. I’m sure that more than one client has taken the approach of asking me to explain where they can learn about best practices. I’m sure that they were somewhat successful. Another approach is to slide the lunch menu in front of them; I have only met one auditor that would not be distracted by a lunch menu.

6 Step Approach When You Disagree With an Auditor

1. Shut-up and look it up (before you open your mouth, grab the applicable external Standard and locate the information you are looking for).

2. If you are still convinced that the auditor is wrong, then tell that you are having trouble finding the requirement. Show them where you are looking, and then ask them to help you find the requirement.

3. If the auditor can’t show you where you are wrong, or it appears that the auditor is interpreting the Standard as they see fit, then focus on asking the auditor for guidance on what they will be looking for in your CAPA plan.

4. If the CAPA plan the auditor is looking for is something you think is a good idea, then shut up and implement the improvements. If the CAPA plan is not acceptable to you, then you should ask what the process is for the resolution of disputes.

5. No matter what, don’t start an argument with the registrar. They enjoy it. They like a challenge and resent people with less experience criticizing them.

6. If you still disagree with your auditor, then you should ask if the auditor can explain the process for appealing findings and follow that process.

A 6 Step Approach if You Disagree With a Notified Body Auditor Read More »

3 Tools for Effectively Qualifying Suppliers

%name 3 Tools for Effectively Qualifying Suppliers
Do you have the right tools for qualifying your suppliers?


For every task, you have a choice of tools that you can use. For qualifying your suppliers, are you using the correct tools? 

This blog reviews how to utilize statistical process control, process validation, and supplier auditing to qualify suppliers effectively.
If you could afford to audition suppliers for a few months against hundreds of other competitors, then only the qualified suppliers would be approved. Unfortunately, you don’t have the same budget that American Idol has. So what should you do instead?

Most companies use the same three, tired tools to qualify suppliers: ISO Certification, Quality Manuals, and questionnaires. ISO certification is a weak tool because certification is only as good as the registrar’s worst client. Quality Manuals are intended to define the intent of your supplier’s Quality Management System, while most of the details are located in procedures. You only need a copy of your supplier’s Quality Manual to help you plan audits. Supplier questionnaires seem to be the most popular tool, but most of the questions require a “Yes/No” response that suppliers rarely answer negatively. To assess the qualifications of potential suppliers more effectively, try using the following tools instead:

Tool # 1: Statistical Process Control

Most companies require a Certificate of Compliance (CoC) with every shipment. A CoC is useless. Just like the “Yes/No” responses to questionnaires, you will never see a CoC that indicates something is wrong. A Certificate of Analysis (CoA) is much more useful, because the CoA has actual data, and the tolerance range is typically indicated for each test or measurement that was performed by the supplier. The best report you can get from a supplier is a statistical analysis of each specification during the prototype production lot. When you have a Statistical Process Control (SPC) run chart, you know quantitatively if the supplier is capable of making an acceptable product. The run chart can also be used to develop an appropriate sampling plan for incoming inspection.

Tool # 2: Process Validation

Process validation is much more than determining if a process is capable of producing a consistent product. An SPC run chart can do that. Process validation tells you what range of operating parameters will produce a consistent product. Therefore, when you have process deviations or measurement devices are slightly out-of-calibration, you will know if your supplier’s process will still make an acceptable product. The validation of a process should also identify which variables are critical indicators of the process. This information can be used to reduce the number of variables and specifications that are monitored for a production process, and focus both your supplier’s resources and your own.

Tool # 3: Supplier Auditing

A multi-disciplinary team audit of a potential supplier is an effective tool for assessing a supplier’s qualifications and will help build a stronger relationship between your team and the supplier’s team. Before you conduct an audit, it is important to plan the audit to ensure you get the greatest possible value. The following recommendations are important to supplier auditing:

  1. Use a risk-based approach to auditing suppliers (this goes beyond just critical and non-critical)
  1. Strategically select auditors and train them well
  2. Plan the auditing goals and objectives for the team in advance
  3. Create a formal audit agenda that defines which processes each auditor will be focusing on

Auditing 100% of your critical suppliers may seem impossible, due to limited resources, but have you ever seen a cost/benefit analysis?

What’s the cost of rejects, rework, and product redesign?

Supplier Quality Management Webinars Available 

Are your Suppliers Qualified? Prove It! 

http://robertpackard.wpengine.com/suppliers-qualified-prove/

Supplier Auditing and Remote Auditing: Tips to Save You Time and Money 

http://robertpackard.wpengine.com/supplier-auditing-and-remote-auditing-tips-save-time-money/

 

 

3 Tools for Effectively Qualifying Suppliers Read More »

The Audit Program Manager: 4 Areas of Auditor Competency

rookie The Audit Program Manager: 4 Areas of Auditor Competency

Passing a webinar on auditing does not make you competent.

This blog reviews an audit program manager’s four areas of auditor competency; experience, skills, training, and education.

Does your company ask incoming inspectors to update CAD drawings when there is a design change? Of course not. Your company has engineers that are trained to use SolidWorks, and it takes a new engineer awhile to become proficient with the software. Auditing is a skill that you learn—just like SolidWorks.

I’ve never met a manager that wondered where the value was in having an engineer update a drawing, but many managers view internal and supplier audits as a necessary evil. Instead of asking the expert how few audit days you can get away with, ask the expert: “What is the purpose of auditing?”

The purpose of internal auditing is to confirm that the management system is effective and identify opportunities for improvement. The purpose of supplier auditing is to verify that a supplier is capable of meeting your needs and identify opportunities for improvement. Therefore, if an auditor has no nonconformities and no opportunities for improvement were identified—what a waste of time!

To receive value from auditing, you need auditors that are competent. In clause 6.2.1 of the ISO 13485 Standard, it states, “Personnel performing work affecting product quality shall be competent based on appropriate education, training, skills, and experience.” As the audit program manager, ensure you recruit people that demonstrate auditing competency.

Education

First, educational background is important for auditors. You cannot expect someone who has never taken a microbiology course in their life to be an effective auditor of sterilization validation. Likewise, someone that has never taken a course in electricity and magnetism will not be effective as an auditor for active implantable devices. Therefore, determine what types of processes the auditor will be auditing. Then ensure that the person you hire to be an auditor has the necessary education to understand the processes they will be auditing.

Training

Second, an auditor needs to be trained before they can audit. The auditor needs training in three different aspects: 1) the process they will be auditing, 2) the standard that is the basis for assessing conformity, and 3) auditing techniques. If you are going to be auditing Printed Circuit Board (PCB) manufacturers with Surface-Mount Technology (SMT), then you need to learn about the types of components used to make PCBs, and how these components are soldered to a raw board. I know first-hand that anyone can learn how SMT works, but it took me a few months of studying.

If your company is only selling medical devices in the United States, then you will need to learn 21 CFR 820 (i.e., – the QSR). However, if your company also sells devices in Europe or Canada, you will need to learn ISO 13485, the Medical Device Directive (MDD) (93/42/EEC as modified by 2007/47/EC), and the Canadian Medical Device Regulations (CMDR). I learned about ISO 13485 in a four-and-a-half day lead auditor course in Florida,  MDD in a three-day CE Marking Course in Virginia, and the CMDR in a two-day course taught by Health Canada in Ontario. A 50-minute webinar on each regulation is not sufficient for auditing.

Finally, you need training in the techniques of auditing. A two-day course is typically needed. I took a 50-minute webinar and passed a quiz before conducting my first internal audit, but I had not developed my skills at that point. 

Skills

Third, an auditor needs communication, organizational, and analytical skills to be useful as an auditor. Communications skills must include the ability to read and write exceptionally well, and the auditor needs to be able to verbally communicate with auditees during meetings and interviews. The most difficult challenge for auditors is covering all items on their agenda in the time available. The auditor rarely has more time than the need to audit any topic, and audit team leaders must be able to manage their own time, as well as simultaneously managing the time of several other auditors. 

Experience

Last, but indeed not the least important aspect of auditor competency, is experience. This is why third-party auditors are required to act as team members under the guidance of a more experienced auditor before they are allowed to perform audits on their own. This is required, regardless of how many internal or supplier audits, the person may have conducted in the past. More experienced auditors are also required to observe new auditors and recommend modifications in their technique. Once a new auditor has completed a sufficient number of audits as a team member, the auditor is then allowed to practice leading audits while being observed. After six to nine months, a new auditor is finally ready to be a lead auditor on their own. An internal auditor does not need the same degree of experience as a third-party auditor, but being shadowed two-three times is not sufficient experience for an auditor (first or second-party). For more information about this topic, please read my blog posting on auditor shadowing.

The Audit Program Manager: 4 Areas of Auditor Competency Read More »

Internal Audit Training for New Hires

 

welcome aboard Internal Audit Training for New Hires

The author discusses a few proven internal audit training strategies (i.e., shadowing, auditing process owners) for new hires.

Once you have identified someone that you want to “hire” as an internal auditor, your next step should be to develop an “Onboarding plan for them with their boss. If you are hiring someone that will be a dedicated auditor, please ignore my quotation marks above. In most companies, however, the internal auditors are volunteers that report to another hiring manager. Therefore, as the audit program manager, you need to get a firm commitment from the auditor’s boss with regard to the time required to train the new auditor and to perform audits on an ongoing basis. 

Winning Over the Boss

In my previous posting, I said that “The biggest reason why you want to be an auditor is that it will make you more valuable to the company.” The auditor’s boss may or may not agree with this statement, but the boss knows that the salary is coming out of their budget either way. Therefore, talk with the auditor’s boss and determine what the auditor’s strengths and weaknesses are. Find out which skills the boss would like to see the auditor develop. By doing this, the two of you can develop a plan for making the auditor more valuable to their boss AND the company. 

Making Re-Introductions

Ideally, auditors are extraverted and have worked at the company long enough to know the processes and process owners that they will be assigned to audit—especially if they will be auditing upstream and downstream from their process area. In the past, the auditor may have been a customer or a supplier, but now the relationship with a process owner will change. Auditors are required to interview process owners, and this involves asking tough questions that might not be appropriate in the auditor’s regular job duties. Therefore, as the audit program manager, you should re-introduce the auditor to the process owner in their new capacity as an auditor. During this re-introduction, it is important to make three points:

  1. The auditor is going to be trained first (on auditing and ISO 13485)
  2. You will be shadowing the auditor during the audit, and
  3. The auditor’s job is to help the process owner identify opportunities for improvement

By making the first point, you are reminding the process owner of the scheduled audit—well in advance. You are also informing the process owner that this auditor will have new skills, and the process owner should have some tolerance for mistakes that new employees make. You might also mention that you would like to get the process owner’s feedback after the audit, so the auditor knows which areas they need to improve upon to become better auditors. The second point should put the process owner at ease—assuming the process owner has a good relationship with you as the audit program manager. It is important to be descriptive when “shadowing” is mentioned. Both the process owner and the auditor may not understand the process or the purpose of shadowing. The following blog posting might help with this: “How do you shadow an auditor? Did you learn anything?”

The third point is the most critical step in onboarding a new auditor. For an auditor to be successful, they must ADD VALUE! As an auditor, you cannot pretend to add value. The process owner should know their process, and they probably know which areas are weakest. The audit program manager should encourage the process owner to list some specific areas in which they are having problems. Ideally, the process owner would be informed of this need before the re-introduction. Then the process owner can be better prepared for the meeting, and hopefully, they will have a few target areas already identified. Targets with associated metrics are the best choice for a new auditor because these targets reinforce the process approach to auditing. 

Next Steps for Internal Audit Training

Once your new auditor has been re-introduced to the process owners, they will be auditing, and you need to begin the training process. As with any new employee, it is important to document training requirements and to assess the auditor’s qualifications against the requirements of an auditor. Every new auditor will need some training, but the training should be tailored specifically to the needs of the auditor. The training plan for a new auditor should include the following:

  1. A reading list of company procedures specific to auditing and external standards that are relevant
  2. Scheduled dates for the auditor to shadow another experienced auditor
  3. Scheduled dates for an experienced auditor to shadow the auditor during the first two process audits (upstream and downstream)
  4. Goals and objectives for the internal audit program; and
  5. Any training goals that the auditor’s boss has identified for the auditor

 

Internal Audit Training for New Hires Read More »

Auditing ISO 14971 – 4 Steps to Assess Compliance

This article describes four key steps for auditing ISO 14971, and suggested auditing questions are included.

Let’s say that you went ahead and purchased ISO 14971:2012, read Annex ZA, and identified a couple of gaps in your procedure. After you revised your Risk Management procedure to be compliant with the revised Standard, then what are you supposed to do?

Most QA Managers struggle over whether they should purchase ISO 14971:2012. I wrote a couple of blog postings about this matter, but my point was not to debate this question but to ensure companies are aware that they need to be compliant with the MDD and the ISO 14971 Standard. The “changes” from 2009 to the 2012 version are simply the European Commission reminding manufacturers that there are seven aspects of the ISO 14791 Standard that do not meet the requirements of the MDD. Therefore, if your company has already verified that your risk management process is compliant with the MDD–then you have nothing to change. However, if your risk management process is only compliant with ISO 14971:2009, then you need to revise your processes and procedures to address these seven aspects. 

4 Steps in Auditing ISO 14971

Once you have made revisions to your risk management process, how do you perform auditing of ISO 14971?

Step 1: Planning your auditing ISO 14971

This will be an internal audit, and since you (the QA Manager) are the process owner for the risk management process, you personally cannot audit this process. You need to assign someone that has the technical skill to perform the audit, but this person cannot be the process owner (you) or a direct report to the process owner (the rest of the QA department). Fortunately, the Director of Engineering is also trained as an internal auditor at your company. She is trained on ISO 14971:2009, but she did not receive risk management training to the most current version. To address this gap, she must read the updated Standard to understand what’s new.

novcover preview 211x300 Auditing ISO 14971   4 Steps to Assess Compliance
Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

She has participated in risk management activities, but each product development engineer participates in risk management activities for their own design projects. Therefore, she has several projects she can sample risk management records from without auditing her own work. You have communicated that you need this audit finished sometime in December because you want any CAPAs resulting from the audit to be finalized before the next Management Review at the end of January. The timing of the Management Review is important because the risk management procedure requires that top management assess the effectiveness of the risk management process during Management Review meetings.

There are no previous audit findings to close from the last audit of the risk management process. Still, the Director of Engineering has seven specific items to emphasize from the 2012 revision of the Standard, and a revised procedure for risk management. Therefore, she will prepare for the audit by identifying some new interview questions to specifically address these changes–as well as some more general, open-ended questions.

Specific questions related to Annex ZA when auditing ISO 14971

1. How does the risk analysis evaluate the acceptability of risks in the lowest category? (This is a leading question, but it is specifically designed to determine if negligible risks are discarded).

2. Please provide a few examples of how risks in the lowest category were reduced. (In sections 1 and 2 of the Annex, I require all risks to be reduced as far as possible, and for all risks to be evaluated for acceptability. The wording of this question also allows auditors flexibility in their sampling).

3.  How did the design team determine when they had implemented sufficient risk controls to minimize risks? (Many companies use a color-coded matrix as a quasi-objective method for determining when risks are adequately reduced. This process is often referred to as the ALARP concept. Annex ZA specifically prohibits using economic considerations as part of this determination).

4. How did you conduct a risk-benefit analysis? (The Standard allows for performing a risk-benefit analysis when overall residual risks exceed the acceptability criteria as outlined in the risk management plan. However, the MDD requires an overall risk-benefit analysis in Section 1 of Annex I. Section 6 also requires that a risk-benefit analysis be performed for each individual risk).

5. How were risk control options selected? (Section 2 of the MDD implies that the manufacturer shall review All the control options and pick the most appropriate ones. Therefore, the auditor should specifically look for evidence that the team systematically reviewed all possible control options to reduce risks–rather than stopping as soon as the risks were reduced to an acceptable level).

6. What were your team’s priorities for the implementation of risk control options? (It’s possible that the previous question will be sufficient to gather evidence that risk controls were implemented with the required prioritization, as specified in the MDD. However, this question would be used as a follow-up question if it is not clear that the team prioritized the risk control options in accordance with Section 2 of Annex I).

7. How was the effect of labeling and warnings in the instructions for use incorporated into the estimation of residual risks? (Almost every company remembers to include residual risks in their IFU as a warning or caution statement. However, Section 2 of Annex I does not allow for including this information given to the users as a method of reducing risks. Therefore, in a Design FMEA, you would not list labeling and IFUs in your column for current risk controls when you determine the risk. This should be identified as an action to be taken–with no impact on the score for residual risk).

%name Auditing ISO 14971   4 Steps to Assess ComplianceThe above questions are not examples of using the process approach, but each question is phrased in an open-ended manner to maximize the objective evidence gathered during the interview process. If you are doing a process audit, it’s still acceptable to include questions that use the element approach.

Generic questions when auditing ISO 14971

1. When was the ISO 14971:2012 version of the Standard added to the controlled list of external Standards?

2. Please provide examples of where you have updated the Essential Requirements Checklist (a Technical File document) to reference the newest revision of ISO 14971:2012, and please show at least one example of how the risk management report was updated to reflect this revision.

3. How did you verify training effectiveness for the design team specific to the updated risk management procedure before conducting a risk analysis?

%name Auditing ISO 14971   4 Steps to Assess ComplianceThese generic questions do not require reading the ISO 14971:2012 Standard. Instead, each question forces the auditee to demonstrate their knowledge of the revised Standard by answering open-ended interview questions. Each of these questions is also designed to test linkages with other support processes. This is an example of how to use the process approach.

Step 2: Auditing ISO 14971

The next step is to conduct your audit of ISO 14971. During the auditing of ISO 14971, the Director of Engineering will gather objective evidence of both conformity and nonconformity for the risk management process. The generic interview questions that were developed allow her to evaluate the effectiveness of linkages between the risk management process and other processes, such as:

1) Document control

2) Creating technical documentation for regulatory submissions

3) The training process

Specific questions verify that each of the seven elements identified in Annex ZA of ISO 14971:2012 is adequately addressed in the revised procedure. When the audit is completed, the auditor will have a closing meeting with the process owner (you) and the auditee(s), so that everyone is clear about what the findings were, and if there were any nonconformities. This is the time to clarify what needs to be done to prevent each nonconformity from recurring.

Step 3: Writing the Report & Taking Corrective Action(s)

This is no different from any other audit. Still, it is critical to have the report completed soon enough so that CAPAs can be initiated (not necessarily completed) before the Management Review.

Step 4: Verifying Effectiveness of Corrective Action(s)

Many people struggle with verifying the effectiveness of corrective actions–regardless of the process. My advice is to identify a process metric to measure effectiveness. Then the effectiveness check is objective. For example, monitoring the frequency of updates to the list of external standards can help verify that the process for monitoring when Standards are updated is effective. Likewise, the frequency of updates to the Essential Requirements Checklist and the risk management records referenced in the Essential Requirements Checklist indicates if the risk management process is being maintained. Finally, monitoring the lag between the time procedures are updated and when the associated training records are updated quickly identifies if there is a systemic problem with training or if a training gap is just an example of a single lapse.

Auditing ISO 14971 – 4 Steps to Assess Compliance Read More »

How to Finish your Audit Schedule by December 31st

This blog provides viable options to consider related to successfully completing your audit schedule by year’s end.

Let’s say that there are 34 days until the end of 2012. You have four supplier audits and three internal audits to complete. Of course, all but two of these ISO 13485 audits are overdue. What should you do?

Options that might be readily available to you include:

  1. Get some help
  2. Perform remote audits
  3. Reschedule some of the audits for next year

There are some great cartoons and jokes about doing more with less, but if you intend to complete seven audits before the end of the year, you might need some help. There really isn’t any time left to train someone, so that they are capable of conducting an effective audit by themselves. I expect to prepare a new auditor to take at least six months before I believe they are ready to work solo. Even if you are less demanding than I am, you still would need time for classroom training and shadowing a couple of audits. Therefore, the best I believe you could hope for is one or two solo audits of the seven you need to complete.

Realistically, your only source of help would be auditors that are already trained and consultants. The last month of the year is historically hectic for everyone–especially quality assurance auditors. Therefore, consultants will not be cheap, and you should commit to any qualified consultants that are available without too much delay (then again, maybe they are available because they are not very good). If you have any in-house auditors that are already trained, do everything you can to get some of their time in the next few weeks.

Remote Audits

Option two is to perform remote audits. This is a viable option for you to justify for a supplier with an impressive quality track record, or suppliers in other countries. However, a remote audit is not the same as asking a supplier to complete a survey. ISO 19011:2011 provides some guidance specific to remote auditing in table B.1 of Annex B.

For a remote audit, you should still sample just as many records—if not more. You should conduct interviews by phone, Skype, or some similar technology. You should analyze any available data to help identify which processes appear to be effective and which processes need to improve. If you are performing a remote audit for the first time, I recommend focusing on the same processes that you would normally audit in a conference room, rather than processes that you would typically audit where they occur—such as production controls. Regardless of which process you check, you should always request data.

Option three is to reschedule some audits for January 2013. I have suggested this so many times to clients, but very few follow this advice. If your company is late in conducting some audits, the important thing to do is to document this, reschedule the audits, and take corrective action(s) to prevent it from recurrence. If you wait until January, you will have additional time to train an auditor, as well. Finally, consultants historically have more time available in January than December.

In parallel with your efforts to catch-up on your schedule, I also recommend the following:

Create a quality objective that measures the “on-time delivery” of audits and audit reports. This is an effective metric for managing an audit program.

Investigate the reasons for audits being overdue. If the occurrence was preventable, then I recommend initiating a CAPA. This will have two effects. First, your third-party auditors will see that you have identified the problem yourself and taken appropriate corrective action(s). If you also discuss this during a Management Review, this information can be used effectively to change the grading of an audit finding to a “minor,” or to potentially eliminate the finding altogether. Second, it will ensure that this doesn’t occur again.

How to Finish your Audit Schedule by December 31st Read More »

Instructor Effectiveness and the Power of a SNICKERS

The author discusses his personal experience attending a training class, instructor effectiveness, and reasons why he learned so much there.

I guess there are still some instructors out there that need to be reminded that we can all read the regulations on our own. We don’t need to pay $1,000+ per day to have someone read stuff for us. If that’s what you want, my 10-year old son is a fantastic reader. He’ll record anything you want, in any media format, for a much smaller dollar figure. If you want to learn something that is worth at least as much as your investment of time and money, then you need to find an instructor that can teach effectively.

Four Prerequisites for a Great Instructor:

1. The instructor must be an expert

2. The instructor must inspire participation

3. The instructor must provide practical examples for each student

4. The instructor must get everyone’s attention–and keep it

The most important determining factor of training effectiveness, however, occurs after the course is over When you are teaching quality assurance and regulatory affairs, you must develop your ability to inspire and engage students to Olympic medalist proportions. “Blah, blah, blah…” and “Death by PowerPoint” will get you fired. Don’t read your slides, don’t turn your back on the audience (or they’ll attack) and PLEASE don’t ever ask someone to read the definition of nonconformity out loud to the rest of the group. When I teach a class, you demand my best. I’m six-foot, six inches tall, and I have a loud booming voice. My mother has red hair, and she was an opera singer. I’ve got the voice to fill any auditorium and stage presence to match. But if you even start to nod off in class, I may just have to throw a Snickers bar at you.

snickers Instructor Effectiveness and the Power of a SNICKERS
This is an essential tool for any instructor. It functions as a tool to prod sleeping students awake, is small enough to cause minimal injury when thrown, serves as an emergency food supply, and is gluten-free.

If legal counsel recommends against using projectiles to encourage class participation, you might also consider one of my all-time genius ideas. I was scheduled for a two-day course in Ottawa, but the day before, I needed to perform an audit in Pennsylvania. Therefore, my flight was the last flight into Ottawa–arriving at approximately 1 a.m. My flight was delayed for more than an hour, and the person in front of me was trying to smuggle an extra carton of smokes into the country. Just before 4 a.m., my taxi arrived at the Albert at Bay Suite Hotel. The class started at 8 a.m. I made it to class on time, and excessive consumption of several pots of black coffee helped get me to lunch. Then my legs started getting a little shaky. Fortunately, there was a convenience store next door that sold my favorite chocolate–the Dark Aero bar! After four of these monstrous doses of cacao, and another pot of coffee, I could have listened to the lecture on the Canadian Medical Device Regulations all night.

aero bar Instructor Effectiveness and the Power of a SNICKERS
Hershey’s copied them, but the result was a mere shadow of Nestle’s greatness. Canadians know how to make junk food, tell a joke, and play hockey!

Lessons Learned

Despite the physical handicap of sleep deprivation, I still learned a ton from my course in Canada. Here’s why:

1. The instructors were experts. Both instructors were regulatory experts and Canadian. Both instructors taught this course twice a year for multiple years, and one of the instructors actually worked for Health Canada.

2. The instructors were blessed with the perfect audience that was hyper-motivated to pass the course. Everyone in the class worked for a Notified Body that had sponsored them to take the course. In order to stay employed and get a raise, I needed to pass that course. If I failed the exam, I had to absorb the cost to travel back to Ottawa and retake the course in February (BRRRR!).

3. Everyone has different experiences, and therefore not every example makes sense to us. Therefore, instructors need to use practical examples that are actionable. In this course, the instructors brought more than a dozen medical devices to the class. We studied the labeling and intended use of each device. Even students from Japan, Europe, and Australia were familiar with some of the products. This was critical because we all needed to be able to identify incorrect Canadian labeling.

4. The greatest asset of all was the humor of the instructor from Health Canada. He was hilarious. He had everyone laughing at his jokes for the entire course. Most of the jokes were not funny enough for a stand-up routine, but this was a mandatory regulatory course on Canadian regulations. Who would even expect a chuckle? Despite the strengths of these instructors, there is only one reason why I know the Canadian Medical Device Regulations (CMDR), as well as I do. I use them every single week.

Some Examples of How I Used the CMDR:

First, I had to audit 162 days for BSI in 2011. Ninety percent of those 162 days were for companies that required a Canadian Medical Device License. Therefore, I started auditing companies to the Canadian regulations immediately after the course. Second, I was also consulting for companies at the same time I was auditing for BSI. Consulting clients hired me to prepare and submit the Canadian Medical Device License Applications for them. I also had to revise and create new procedures specific to Canadian regulations. I spent another 60+ days in 2011 doing consulting. Finally, I was one of BSI’s instructors that taught the regulatory comparison course, which compared the regulations of the USA, Canada, Europe, Australia, and Japan.

Therefore, at least once a month, I had a classroom of 6-20 people asking me challenging questions about how to interpret and apply regulations from each of these countries to their products. I used every bit of knowledge I learned in that course in Ottawa, and I started using that knowledge immediately after the course. I had peers, superiors, clients, and students challenging my knowledge of these topics every day. This is what makes you a subject matter expert. If you need to learn something about Quality Assurance or Regulatory Affairs, a one-hour webinar, reading a blog, taking a five-day, or shadowing another more experienced person is not enough. In the end, all of the above will get you to the level of barely competent!  If you want to learn, you need a great instructor. Then you need to use everything you learned at every opportunity for several years. Some say, “If you can’t do, teach.” I say, “Bring a SNICKERS bar and throw it at them for faking it.”

Instructor Effectiveness and the Power of a SNICKERS Read More »

Scroll to Top