ISO Auditing

Internal Auditing, Supplier Auditing, Lead Auditor, Internal Auditor, ISO 19011, and ISO 17021.

Instructor Effectiveness and the Power of a SNICKERS

1 Comment / ISO Auditing / By / , ,

The author discusses his personal experience attending a training class, instructor effectiveness, and reasons why he learned so much there.

I guess there are still some instructors out there that need to be reminded that we can all read the regulations on our own. We don’t need to pay $1,000+ per day to have someone read stuff for us. If that’s what you want, my 10-year old son is a fantastic reader. He’ll record anything you want, in any media format, for a much smaller dollar figure. If you want to learn something that is worth at least as much as your investment of time and money, then you need to find an instructor that can teach effectively.

Four Prerequisites for a Great Instructor:

1. The instructor must be an expert

2. The instructor must inspire participation

3. The instructor must provide practical examples for each student

4. The instructor must get everyone’s attention–and keep it

The most important determining factor of training effectiveness, however, occurs after the course is over When you are teaching quality assurance and regulatory affairs, you must develop your ability to inspire and engage students to Olympic medalist proportions. “Blah, blah, blah…” and “Death by PowerPoint” will get you fired. Don’t read your slides, don’t turn your back on the audience (or they’ll attack) and PLEASE don’t ever ask someone to read the definition of nonconformity out loud to the rest of the group. When I teach a class, you demand my best. I’m six-foot, six inches tall, and I have a loud booming voice. My mother has red hair, and she was an opera singer. I’ve got the voice to fill any auditorium and stage presence to match. But if you even start to nod off in class, I may just have to throw a Snickers bar at you.

snickers Instructor Effectiveness and the Power of a SNICKERS
This is an essential tool for any instructor. It functions as a tool to prod sleeping students awake, is small enough to cause minimal injury when thrown, serves as an emergency food supply, and is gluten-free.

If legal counsel recommends against using projectiles to encourage class participation, you might also consider one of my all-time genius ideas. I was scheduled for a two-day course in Ottawa, but the day before, I needed to perform an audit in Pennsylvania. Therefore, my flight was the last flight into Ottawa–arriving at approximately 1 a.m. My flight was delayed for more than an hour, and the person in front of me was trying to smuggle an extra carton of smokes into the country. Just before 4 a.m., my taxi arrived at the Albert at Bay Suite Hotel. The class started at 8 a.m. I made it to class on time, and excessive consumption of several pots of black coffee helped get me to lunch. Then my legs started getting a little shaky. Fortunately, there was a convenience store next door that sold my favorite chocolate–the Dark Aero bar! After four of these monstrous doses of cacao, and another pot of coffee, I could have listened to the lecture on the Canadian Medical Device Regulations all night.

aero bar Instructor Effectiveness and the Power of a SNICKERS
Hershey’s copied them, but the result was a mere shadow of Nestle’s greatness. Canadians know how to make junk food, tell a joke, and play hockey!

Lessons Learned

Despite the physical handicap of sleep deprivation, I still learned a ton from my course in Canada. Here’s why:

1. The instructors were experts. Both instructors were regulatory experts and Canadian. Both instructors taught this course twice a year for multiple years, and one of the instructors actually worked for Health Canada.

2. The instructors were blessed with the perfect audience that was hyper-motivated to pass the course. Everyone in the class worked for a Notified Body that had sponsored them to take the course. In order to stay employed and get a raise, I needed to pass that course. If I failed the exam, I had to absorb the cost to travel back to Ottawa and retake the course in February (BRRRR!).

3. Everyone has different experiences, and therefore not every example makes sense to us. Therefore, instructors need to use practical examples that are actionable. In this course, the instructors brought more than a dozen medical devices to the class. We studied the labeling and intended use of each device. Even students from Japan, Europe, and Australia were familiar with some of the products. This was critical because we all needed to be able to identify incorrect Canadian labeling.

4. The greatest asset of all was the humor of the instructor from Health Canada. He was hilarious. He had everyone laughing at his jokes for the entire course. Most of the jokes were not funny enough for a stand-up routine, but this was a mandatory regulatory course on Canadian regulations. Who would even expect a chuckle? Despite the strengths of these instructors, there is only one reason why I know the Canadian Medical Device Regulations (CMDR), as well as I do. I use them every single week.

Some Examples of How I Used the CMDR:

First, I had to audit 162 days for BSI in 2011. Ninety percent of those 162 days were for companies that required a Canadian Medical Device License. Therefore, I started auditing companies to the Canadian regulations immediately after the course. Second, I was also consulting for companies at the same time I was auditing for BSI. Consulting clients hired me to prepare and submit the Canadian Medical Device License Applications for them. I also had to revise and create new procedures specific to Canadian regulations. I spent another 60+ days in 2011 doing consulting. Finally, I was one of BSI’s instructors that taught the regulatory comparison course, which compared the regulations of the USA, Canada, Europe, Australia, and Japan.

Therefore, at least once a month, I had a classroom of 6-20 people asking me challenging questions about how to interpret and apply regulations from each of these countries to their products. I used every bit of knowledge I learned in that course in Ottawa, and I started using that knowledge immediately after the course. I had peers, superiors, clients, and students challenging my knowledge of these topics every day. This is what makes you a subject matter expert. If you need to learn something about Quality Assurance or Regulatory Affairs, a one-hour webinar, reading a blog, taking a five-day, or shadowing another more experienced person is not enough. In the end, all of the above will get you to the level of barely competent!  If you want to learn, you need a great instructor. Then you need to use everything you learned at every opportunity for several years. Some say, “If you can’t do, teach.” I say, “Bring a SNICKERS bar and throw it at them for faking it.”

Auditing Design Controls – 7 Step Process

4 Comments / Design Control, ISO Auditing / By / , ,

This blog reviews seven steps for effectively auditing design controls utilizing the ISO 13485 standard and process approach to auditing.

turtle diagram for design controls Auditing Design Controls 7 Step Process

Third-party auditors (i.e., – a Notified Body Auditor) don’t always practice what we preach. I know this may come as a huge shock to everyone, but sometimes we don’t use the process approach. Auditing design controls is a good example of my own failure to follow was it true and pure. Instead, I use NB-MED 2.5.1/rec 5 as a checklist, and I sample Technical Files to identify any weaknesses. The reason I do this is that I want to provide as much value to the auditing client as possible without falling behind in my audit schedule.

Often, I would sample a new Technical File for a new product family that had not been sampled by the Technical Reviewer yet. My reason for doing this is that I could often find elements that are missing from the Technical File before the Technical Reviewer saw the file. This gives the client an opportunity to fix the deficiency before submission and potentially shortens the approval process. Since NB-MED documents are guidance documents, I could not write the client up for a nonconformity, unless they were missing a required element of the M5 version of the MDD (93/42/EEC as modified by 2007/47/EC). This is skirting the edge of consulting for a third- party reviewer, but I found it was a 100% objective way to review Technical Files. I also found I could review an entire Technical File in about an hour.

What’s wrong with this approach to auditing design controls?

This approach only tells you if the elements of a Technical File are present, but it doesn’t evaluate the design process. Therefore, I supplemented my element approach with a process audit of the design change process by picking a few recent design changes that I felt were high-risk issues. During the process audit of the design change process, I sampled the review of risk management documentation, any associated process validation documentation, and the actual design change approval records. If I had time, I looked for the following types of changes: 1) vendor change, 2) specification change, and 3) process change. By doing this, I covered the following clauses in ISO 13485:2016: 7.4 (purchasing), 7.3.9 (design changes), 7.5.6 (process validation), 7.1 (risk management), and 4.2.5 (control of records).

So what is my bastardized process approach to auditing design controls missing? Clauses 7.3.1 through 7.3.10 of ISO 13485:2016 are missing. These clauses are the core of the design and development process. To address this, I would like to suggest the following process approach:

Step 1 – Define the Design Process

Identify the process owner and interview them. Do this in their office–not in the conference room. Get your answers for steps 2-7 directly from them. Ask lots of open-ended questions to prevent “yes/no” responses.

Step 2 – Process Inputs

Identify how design projects are initiated. Look for a record of a meeting where various design projects were vetted and approved for internal funding. These are inputs into the design process. There should be evidence of customer focus, and some examples of corrective actions taken based upon complaints or service trend analysis.

Step 3 – Process Outputs

Identify where Design History Files (DHF) are stored physically or electronically, and determine how the DHF is updated as the design projects progress.

Step 4 – What Resources

This is typically the step of a process audit where their auditor needs to identify “what resources” are used in the process. However, only companies that have software systems for design controls have resources dedicated to Design and Development. I have indicated this in the “Turtle Diagram” presented above.

Step 5 – With Whom, Auditing Training Records

Identify which people are assigned to the design team for a design project. Sometimes companies assign great teams. In this case, the auditor should focus on the team members that must review and approve design inputs (see Clause 7.3.2) and design outputs (see Clause 7.3.3). All of these team members should have training records for Design Control procedures and Risk Management procedures.

Step 6 – Auditing Design Controls Procedures and Forms

Identify the design control procedures and forms. Do not read and review these procedures. Auditors never have the time to do this. Instead, ask the process owner to identify specific procedures or clauses within procedures where clauses in the ISO Standard are addressed. If the process owner knows exactly where to find what you are looking for, they’re training was effective, or they may have written the procedure(s). If the process owner has trouble locating the clauses you are requesting, spend more time sampling training records.

Step 7 – Process Metrics

Ask the process owner to identify some metrics or quality objectives they are using to monitor and improve the design and development process. This is a struggle for many process owners–not just design. If any metrics are not performing up to expectations, there should be evidence of actions being taken to address this. If no metrics are being tracked by the process owner, you might review schedule compliance.

Many design projects are behind schedule, and therefore this is an important metric for most companies. Now that you have completed your “Turtle Diagram,” if you have more time to audit the design process, you can interview team members to review their role in the design process. You could also sample-specific Technical Files as I indicated above. If you are performing a thorough internal audit, I recommend doing both. To learn more about using the process approach to auditing, you can register for our webinar on the topic.

Auditing Medical Device Software Vendors

1 Comment / ISO Auditing / By / ,

This blog presents some thoughts related to auditing medical device software companies.

Software medical devices are used to assist medical professionals. For example, radiologists use software with identifying areas of interest for medical imaging. Do you know how to audit a software company?

As a third-party auditor, I have had the pleasure of auditing software companies for CE Marking. When you audit a software company for the first time, this forces you to re-learn the entire ISO 13485 Standard. For example, if a company only produces software (i.e. software as a medical device or SaMD) there is very little to sample for incoming inspection and purchasing records. This is because the product is not physical—it’s software. Clauses of ISO 13485 related to sterility, implants, and servicing are also not applicable to software products. If the software is web-based, the shipping and distribution clauses (i.e., – 7.5.5) might present a challenge to an auditor as well.

The aspects of the ISO 13485 Standard that I found to be the most important to auditing software products were design controls and customer communication. Many auditors are trained in auditing the design and development of software, but very few auditors have experience auditing technical support call centers. When auditing a call center, most of the calls represent potential complaints related to software “bugs,” system incompatibilities with the operating system or hardware, and use errors resulting from the design of the user interface.

In most technical support call centers, the support person tries to find a work-around for problems that are identified. The problem with a “work-around” is that it is the opposite approach to the CAPA process. To meet ISO 13485 requirements, software companies must show evidence of monitoring and measuring these “bugs.” There must also be evidence of management identifying negative trends and implementing corrective actions when appropriate.

As an auditor, you should focus on how the company prioritizes “bugs” for corrective actions. Most software companies focus on the severity of software operations and the probability of occurrence. This is the wrong approach. Failure to operate is not the most severe result of medical device software failure. Medical device software can result in injury or death to patients. Therefore, it is critical to use a risk-based approach to the prioritization of CAPAs. This risk-based approach should focus on the severity of effects upon patients—not users. This focus on safety and performance is emphasized throughout the EU Medical Device Regulations and it is a risk management requirement in ISO 14971.

Process Approach to Auditing – 7 Steps to Training Auditors

4 Comments / ISO Auditing / By / , ,

The process approach to auditing is demonstrated using Turtle Diagrams as a tool instead of using traditional auditor checklists.

tutle diagram1 Process Approach to Auditing 7 Steps to Training Auditors

I have been reviewing trends for how people find my website, and a large number of you appear to be interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.

First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead, you audit each process. Typical processes include:
  1. Design & Development
  2. Purchasing
  3. Incoming inspection
  4. Assembly
  5. Final Inspection
  6. Packaging
  7. Sterilization
  8. Customer Service
  9. Shipping
  10. Management Review
  11. CAPA
  12. Internal Auditing

Why the Process Approach is Recommended

First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments, the process approach will expose it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked.

Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e.,– the element approach). My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are incorporated into each process audit.

For example, each process audit requires a review of records as input and outputs. Also, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment. The tool that BSI uses to teach the process approach is the “Turtle Diagram.” The diagram above illustrates where the name came from.

Interviewing with the Process Approach

The first skill to teach a new auditor is the interview. Each process approach audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next, most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique because it is a “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore, I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.

After getting a general overview, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.

The next step of this approach is to “determine what resources are used by incoming inspection.” This includes gauges used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list, the auditor should select people to interview and follow-up with a request for training records.

The sixth step is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures, this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

Challenging Process Owners

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company quality objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review the monitoring and measurement of processes, and the trend analysis can be verified to be input into the CAPA process.

In my brief description of the process approach, I used the incoming inspection process. I typically choose this process for training new auditors because it is a process that is quite similar in almost every company, and it is easy to understand. More importantly, however, the incoming inspection process does an effective job of covering more clauses of the Standard than most audits. Therefore, new auditors get an appreciation for how almost all the clauses can be addressed in one process audit. If you are interested in learning more about Turtle Diagrams and the process approach to auditing, please register for our webinar on the process approach to auditing.

Scroll to Top