Blog

Archive for ISO Auditing

How to Finish your Audit Schedule by December 31st

This blog provides viable options to consider related to successfully completing your audit schedule by year’s end.

Let’s say that there are 34 days until the end of 2012. You have four supplier audits and three internal audits to complete. Of course, all but two of these audits are overdue. What should you do?

Options that might be readily available to you include:

  1. Get some help
  2. Perform remote audits
  3. Reschedule some of the audits for next year

There are some great cartoons and jokes about doing more with less, but if you intend to complete seven audits before the end of the year, you might need some help. There really isn’t any time left to train someone, so that they are capable of conducting an effective audit by themselves. In fact, I expect training a new auditor to take at least six months before I believe they are ready to work solo. Even if you are less demanding than I am, you still would need time for classroom training and shadowing a couple of audits. Therefore, the best I believe you could hope for is one or two solo audits of the seven you need to complete.

Realistically, your only source of help would be auditors that are already trained and consultants. The last month of the year is historically very busy for everyone–especially quality assurance auditors. Therefore, consultants will not be cheap, and you should commit to any qualified consultants that are available without too much delay (then again, maybe they are available because they are not very good). If you have any in-house auditors that are already trained, do everything you can to get some of their time in the next few weeks.

Remote Audits

Option two is to perform remote audits. This is a viable option for you to justify for a supplier with an impressive quality track record, or suppliers in other countries. However, a remote audit is not the same as asking a supplier to complete a survey. ISO 19011:2011 provides some guidance specific to remote auditing in table B.1 of Annex B.

For a remote audit, you should still sample just as many records—if not more. You should conduct interviews by phone, Skype or some similar technology. You should analyze any available data to help identify which processes appear to be effective and which processes need to improve. If you are performing a remote audit for the first time, I recommend focusing on the same processes that you would normally audit in a conference room, rather than processes that you would typically audit where they occur—such as production controls. Regardless of which process you check, you should always request data.

Option three is to reschedule some audits for January 2013. I have suggested this so many times to clients, but very few follow this advice. If your company is late in conducting some audits, the important thing to do is to document this, reschedule the audits, and take corrective action(s) to prevent it from recurrence. If you wait until January, you will have additional time to train an auditor, as well. Finally, consultants historically have more time available in January than December.

In parallel with your efforts to catch-up on your schedule, I also recommend the following:

Create a quality objective that measures “on-time delivery” of audits and audit reports. This is an effective metric for managing an audit program.

Investigate the reasons for audits being overdue. If the occurrence was preventable, then I recommend initiating a CAPA. This will have two effects. First, your third-party auditors will see that you have identified the problem yourself and taken appropriate corrective action(s). If you also discuss this during a Management Review, this information can be used effectively to change the grading of an audit finding to a “minor,” or to potentially eliminate the finding altogether. Second, it will ensure that this doesn’t occur again.

Posted in: ISO Auditing

Leave a Comment (0) →

“Shadowing” as an Effective Auditor Quality Training Technique

%name Shadowing as an Effective Auditor Quality Training Technique

If you are shadowing, you are taking notes, so you can discuss your observations with the person you are shadowing later.

This article reviews shadowing as an effective auditor quality training technique in a first, second and third-party audit. Also, five common shadowing mistakes trainers make is discussed as well.

Somewhere in your procedure for “Quality Audits,” I’ll bet there is a section on auditor competency. Most companies require that the auditor has completed either a course for internal auditor or a lead auditor course. If the course had an exam, then you might even have evidence for training effectiveness. Demonstrating competency is much harder. One way is to review internal audit reports, but writing reports is just part of what an auditor does. How can you evaluate an auditor’s ability to interview people, take notes, follow audit trails and manage their time? The most common solution is to require that the auditor “shadow” a more experienced auditor several times, and then the trainee will be “shadowed” by the trainer. 

Shadowing 1st Party Audits

ISO 19011:2011 defines first-party audits as internal audits. When first-party auditors are being shadowed by a trainer, or vice versa, there are many opportunities for training. The key to successful training of auditors is to recognize teachable moments.

When the trainer is auditing, the trainer should look for opportunities to ask the trainee, “What should I do now?” or “What information do I need to record?” In these situations, the trainer is asking the trainee what they should do BEFORE they do it. If the trainee is unsure, the trainer should explain what, why and how at that moment with real examples.

When the trainer is shadowing, the trainer should watch and wait for a missed opportunity to gather important information. In these situations, the trainer must resist guiding the trainee until after the trainee appears to be done. When it happens, sometimes the best tool is simply asking, “Are you sure you got all the information you came for?”

top5 Shadowing as an Effective Auditor Quality Training Technique Here are five (5) mistakes that I have observed trainers make when they were shadowing:

1. Splitting up, instead of staying together, is one of the more common mistakes I have observed. This happens when people are more interested in completing an audit, rather than taking every advantage of training opportunities. The trainee may be capable of auditing on their own, but this is unfair to the trainee, because they need feedback on their auditing technique. This is also unfair to the auditee, because it is extremely difficult to support multiple auditors simultaneously, and when it is unplanned, there may not be trainers available for both auditors. If an audit is running behind schedule, this is the perfect time to teach a trainee how to recover some time in their schedule. Time management is after all one of the hardest skills for auditors to master.

2. Staying in the conference room, instead of going to where the work is done, is a common criticism of auditors. If the information you need to audit can be found in a conference room, then you could have completed the audit remotely. This type of audit teaches new auditors very little, other than how to take notes. These are basic skills that auditors should master in a classroom prior to shadowing.

3. Choosing an administrative process is a mistake, because administrative processes limit the number of aspects of the process approach that can be practiced by an auditor-in-training. Administrative processes rarely have equipment that requires validation or calibration, and both the process inputs and outputs consist only of paperwork, forms or computer records. With raw materials and finished goods to process, the job of the auditor is more challenging, because there is more to be aware of.

4. Not providing honest feedback is a huge mistake. Auditors need to be thick skinned, or they don’t belong in a role where they are going to criticize others. Before you begin telling other people how to improve, you first need to self-reflect and identify your own strengths and weaknesses. Understanding your own perspective, strengths, weaknesses and prejudices is critical to being an effective assessor. As a trainer, it is your job to help new auditors to self-reflect and accurately rate their performance against objective standards.

5. “Silent Shadowing” has no value at all. By this I mean shadowing another auditor without asking questions. If you are a trainee, you should be mentally pretending you are doing the audit. Whenever the trainer does something different from the way you would do things, you should make a note so you can ask, “Why did you do that?” If you are the trainer, you should also be mentally pretending you are doing the audit. It is not enough to be present. Your job is to identify opportunities for the trainee to improve. The better the trainee, the more challenging it becomes to identify areas for improvement.  This is why training other auditors has helped me improve my own auditing skills.

Shadowing Second-Party Audits

supply chain weakest link Shadowing as an Effective Auditor Quality Training Technique

If you are developing a new supplier quality engineer that is responsible for performing supplier audits, it is recommended to observe the auditor during some actual supplier audits. Supplier audits are defined as second-party audits in the ISO 19011:2011 Standard. The purpose of these audits is not to verify conformity to all the aspects of ISO 13485. Instead, the primary purpose of these audits is to verify that the supplier has adequate controls in place to consistently manufacture conforming product for your company. Therefore, processes such as Management Review (Clause 5.6) and Internal Auditing (Clause 8.2.2) are not typically sampled during a second-party audit.

The two most valuable processes for a second-party auditor to sample are: 1) incoming inspection, and 2) production controls. Using the process approach to auditing, the second-party auditor will have an opportunity to verify that the supplier has adequate controls for documents and records for both of these processes. Training records for personnel performing these activities can be sampled. The adequacy of raw material storage can be evaluated by following the flow of accepted raw materials leaving the incoming inspection area. Calibration records can be sampled by gathering equipment numbers from calibrated equipment in use by both processes. Even process validation procedures can be assessed by comparing the actual process parameters being used in manufacturing with the documented process parameters in the most recent validation or re-validation reports.

My recommendation is to have the trainee shadow the trainer during the process audit of the incoming inspection process, and for the trainer to shadow the trainee during the process audit of production processes. In between the two process audits, the trainee should be asking questions to help them fully understand the process approach to auditing. Supplier auditors should also be coached on techniques for overcoming resistance to observe processes that may involve trade secrets, or where competitor products may also be present. During the audit of production processes, the trainer may periodically prompt the trainee to gather information that will be needed for following audit trails to calibration records, document control, or for comparison with the validated process parameters. The “teachable moment” is immediately after the trainee missed an opportunity, but while the trainee is still close enough to go back and capture the missing details.

Shadowing Third-Party Audits

qsit inspection Shadowing as an Effective Auditor Quality Training Technique

Use your FDA inspections and ISO certification audits as an opportunity to shadow experienced auditors and to learn what they are looking for.

If you are going to shadow a third-party auditor, I recommend two specific people to “shadow” the auditor. First, the process owner should be the guide for whichever process is being audited. This is the person that will be responsible for addressing any nonconformities found in the area, and they should be present during interviews–although they should be coached on when to comment and when to remain quiet and simply observe.  Second, the person that performed an internal audit of the process being audited should be present if at all possible. This person will benefit from seeing how a professional third-party auditor performs a process audit, because they will know which things to look for in the future, so that auditees in that area are prepared for the next external audit.

For other sources of information related to auditor shadowing, please check out the following links:

1. Internal Auditor Training – Shadowing external auditor? – from Elsmar Cove

2. Developing Supplier Quality Auditor Training Programs – by Seth Mailhot at NixonPeabody

Posted in: ISO Auditing

Leave a Comment (0) →

Instructor Effectiveness and the Power of a SNICKERS

The author discusses his personal experience attending a training class, instructor effectiveness and reasons why he learned so much there.

I guess there are still some instructors out there that need to be reminded that we can all read the regulations on our own. We don’t need to pay $1,000+ per day in order to have someone read stuff for us. If that’s what you want, my 10-year old son is a fantastic reader. He’ll record anything you want, in any media format, for a much smaller dollar figure. If you want to learn something that is worth at least as much as your investment of time and money, then you need to find an instructor that can teach effectively.

Four Prerequisites for a Great Instructor:

1. The instructor must be an expert

2. The instructor must inspire participation

3. The instructor must provide practical examples for each student

4. The instructor must get everyone’s attention–and keep it

The most important determining factor of training effectiveness, however, occurs after the course is over When you are teaching quality assurance and regulatory affairs, you must develop your ability to inspire and engage students to Olympic medalist proportions. “Blah, blah, blah…” and “Death by PowerPoint” will get you fired. Don’t read your slides, don’t turn your back on the audience (or they’ll attack) and PLEASE don’t ever ask someone to read the definition of nonconformity out loud to the rest of the group. When I teach a class, you demand my best. I’m six-foot, six inches tall and I have a loud booming voice. My mother has red hair and she was an opera singer. I’ve got the voice to fill any auditorium and stage presence to match. But if you even start to nod off in class, I may just have to throw a Snickers bar at you.

snickers Instructor Effectiveness and the Power of a SNICKERS

This is an essential tool for any instructor. It functions as a tool to prod sleeping students awake, is small enough to cause minimal injury when thrown, serves as an emergency food supply and is gluten free.

If legal counsel recommends against using projectiles to encourage class participation, you might also consider one of my all-time genius ideas. I was scheduled for a two-day course in Ottawa, but the day before I needed to perform an audit in Pennsylvania. Therefore, my flight was the last flight into Ottawa–arriving at approximately 1 a.m. My flight was delayed more than an hour, and the person in front of me was trying to smuggle an extra carton of smokes into the country. Just before 4 a.m. my taxi arrived at the Albert at Bay Suite Hotel. Class started at 8 a.m. I made it to class on time, and excessive consumption of several pots of black coffee helped get me to lunch. Then my legs started getting a little wobbly. Fortunately, there was a convenience store next door that sold my favorite chocolate–the Dark Aero bar! After four of these monstrous doses of cacao, and another pot of coffee, I could have listened to the lecture on Canadian Medical Device Regulations all night.

aero bar Instructor Effectiveness and the Power of a SNICKERS

Hershey’s copied them, but the result was a mere shadow of Nestle’s greatness. Canadians know how to make junk food, tell a joke and play hockey!

Lessons Learned

Despite the physical handicap of sleep deprivation, I still learned a ton from my course in Canada. Here’s why:

1. The instructors were experts. Both instructors were regulatory experts and Canadian. Both instructors taught this course twice a year for multiple years, and one of the instructors actually worked for Health Canada.

2. The instructors were blessed with the perfect audience that was hyper-motivated to pass the course. Everyone in the class worked for a Notified Body that had sponsored them to take the course. In order to stay employed and get a raise, I needed to pass that course. If I failed the exam, I had to absorb the cost to travel back to Ottawa and retake the course in February (BRRRR!).

3. Everyone has different experiences and therefore not every example makes sense to us. Therefore, instructors need to use practical examples that are actionable. In  this course, the instructors brought more than a dozen medical devices to the class. We studied the labeling and intended use of each device. Even students from Japan, Europe and Australia were familiar with some of the products. This was critical, because we all needed to be able to identify incorrect Canadian labeling.

4. The greatest asset of all was the humor of the instructor from Health Canada. He was hilarious. He had everyone laughing at his jokes for the entire course. Most of the jokes were not funny enough for a stand-up routine, but this was a mandatory regulatory course on Canadian regulations. Who would even expect a chuckle. Despite the strengths of these instructors, there is only one reason why I know the Canadian Medical Device Regulations (CMDR) as well as I do. I use them every single week.

Some Examples of How I Used the CMDR:

First, I had to audit 162 days for BSI in 2011. Ninety percent of those 162 days were for companies that required a Canadian Medical Device License. Therefore, I started auditing companies to the Canadian regulations immediately after the course. Second, I was also consulting for companies at the same time I was auditing for BSI. Consulting clients hired me to prepare and submit Canadian Medical Device License Applications for them. I also had to revise and create new procedures specific to the Canadian regulations. I spent another 60+ days in 2011 doing consulting. Finally, I was one of BSI’s instructors that taught the regulatory comparison course, which compared the regulations of the USA, Canada, Europe, Australia and Japan.

Therefore, at least once a month, I had a classroom of 6-20 people asking me challenging questions about how to interpret and apply regulations from each of these countries to their products. I used every bit of knowledge I learned in that course in Ottawa and I started using that knowledge immediately after the course. I had peers, superiors, clients and students challenging my knowledge of these topics every day. This is what makes you a subject matter expert. If you need to learn something about Quality Assurance or Regulatory Affairs, a one-hour webinar, reading a blog, taking a five-day, or shadowing another more experienced person is not enough. In the end, all of the above will get you to the level of barely competent!  If you want to learn, you need a great instructor. Then you need to use everything you learned at every opportunity for several years. Some say, “If you can’t do, teach.” I say, “Bring a SNICKERS bar and throw it at them for faking it.”

Posted in: ISO Auditing

Leave a Comment (0) →

Attention Auditors! – Have you Read ISO 19011?

This blog reviews some additions and changes to ISO 19011, which covers the topic of quality management system auditing.

%name Attention Auditors! – Have you Read ISO 19011?

If you have ever taken a lead auditor course, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Quality Management Systems.” In November 2011, this standard was updated and the changes were not superficial.

ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting both internal and external audits and how to determine auditor competency. Improvements to the New 2011 Version of the Standard include:

  1. Broadening the scope to all management systems
  2. Clarifying the relationship between ISO 17021 and ISO 19011
  3. Introduction of remote audit methods
  4. Introduction of risk as an auditing concept
  5. Confidentiality is a “new” principle
  6. Clause 5, Managing an audit program, was reorganized
  7. Clause 6, Performing an audit, was reorganized
  8. Clause 7, Competence and evaluation of auditors, was reorganized & strengthened
  9. Annex B is new and the contents of the help boxes was moved to this Annex
  10. Annex A now includes examples of discipline-specific knowledge and skills

One of the most common points of confusion in the lead auditor course is the difference between first, second and third-party audits. In the previous revision of this Standard, this was just a note at the bottom of page one and the top of page two. The note was not very clear either. The new version of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear:

19011 table 11 Attention Auditors! – Have you Read ISO 19011?

The above table is just an example of the improvements made to ISO 19011, and of course, there is little value-add to clarifying a definition. Figure 1 from the new version, “Process flow for the management of an audit program”, is a better example of a “value-add”. This vertical flow chart is reminiscent of Figure 1 from ISO 14971:2007. It categorizes the various stages of audit program management into the Plan-Do-Check-Act (PDCA) cycle. I highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard. Unfortunately, Figure 2, “Typical audit activities,” does not categorize the stages of audit activities (Clauses 6.2 – 6.7 of the revised Standard) into the PDCA cycle. I guess they needed to leave some improvement for the next revision.

The new version retained the opening meeting checklist that was in the previous revision (Clause 6.4.2), and Clause 6.4.9 has a brief closing meeting checklist. Figure 3, “Overview of the process of collecting and verifying information,” is a poor example of a flow chart. Should I make a better one? (Send me an email if you think I should.)

The most valuable changes in this revision are Clause 5.3.2, “Competence of the person managing the audit program,” and all of Clause 7. Most of the audit procedures I read neglect to define the qualifications and method for determining competency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures I read include qualifications for a “Lead Auditor,” but I seldom see anything regarding competency. Unfortunately, this Standard only specifically addresses “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When I teach people how to be a lead auditor, I spend more than an hour on this topic alone. 

ISO 19011 Standard

The Standard would be more effective by providing an example of how third-party auditors become qualified as a Lead Auditor. Third-party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meeting, conducting the audit, closing meeting and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e., – Stage 2 certification or recertification), and another qualified lead auditor must evaluate you and provide feedback.

The last big additions to this Standard were the Appendices. Annex A provides examples of discipline-specific knowledge and skills of auditors. This section is a little on the boring side. I prefer to tell a story about the internal auditor that was auditing incoming inspection—but they had no idea how to check for calibration, or how to measure components.

Appendix B, the finale, has a table (Table B.1) that provides some guidance on how to conduct remote audits (i.e., – desktop audits). I was pleased to see that conducting interviews is a major part of remote auditing in this table. Section B.7 provides some suggestions with regard to conducting interviews, but if you exhibit all 13 of the professional behavior traits found in Clause 7.2.2, then you really don’t need any advice on how to speak with people. For the rest of us mortals, we could use a five day course on interviewing alone.

Additional guidelines are available on the ISO 19011 Auditing Group website.

Posted in: ISO Auditing

Leave a Comment (0) →

Auditing Medical Device Software Vendors

This blog presents some thoughts related to auditing medical device software companies.

Software medical devices are used to assist medical professionals. For example, radiologists use software with identifying areas of interest for medical imaging. Do you know how to audit a software company?

As a third-party auditor, I have had the pleasure of auditing software companies for CE Marking. When you audit a software company for the first time, this forces you to re-learn the entire ISO 13485 Standard. For example, if a company only produces software, there is very little to sample for incoming inspection and purchasing records. This is because the product is not physical—it’s software. Clauses of ISO 13485 related to sterility, implants and servicing are also not applicable to software products. If the software is web-based, the shipping and distribution clauses (i.e, – 7.5.5) might present a challenge to an auditor as well.

The aspects of the ISO 13485 Standard that I found to be the most important to auditing software products were design controls and customer communication. Many auditors are trained on auditing the design and development of software, but very few auditors have experience auditing technical support call centers. When auditing a call center, most of the calls represent potential complaints related to software “bugs,” system incompatibilities with the operating system or hardware and use errors resulting from the design of the user interface.

In most technical support call centers, the support person tries to find a work-around for problems that are identified. The problem with a “work-around” is that it is the opposite approach to the CAPA process. In order to meet ISO 13485 requirements, software companies must show evidence of monitoring and measuring these “bugs.” There must also be evidence of management identifying negative trends and implementing corrective actions when appropriate.

As an auditor, you should focus on how the company prioritizes “bugs” for corrective actions. Most software companies focus on the severity to software operations and the probability of occurrence. This is the wrong approach. Failure to operate is not the most severe result of medical device software failure. Medical device software can result in injury or death to patients. Therefore, it is critical to use a risk-based approach to prioritization of CAPAs. This risk-based approach should focus upon severity of effects upon patients—not users. This focus on safety and efficacy is an essential requirement of the Medical Device Directive (93/42/EEC as modified by 2007/47/EC) and is a requirement of ISO 14971:2007.

Posted in: ISO Auditing

Leave a Comment (0) →

7 Steps to Training Auditors on the Process Approach of the ISO Standard

The author uses the turtle diagram as the foundation to reviewing seven (7) steps to training auditors on the process approach for the ISO standard.

I have been reviewing trends for how people find my website, and a large number of you appear to be interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.

First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead you audit each process. Typical processes include:
  1. Design & Development
  2. Purchasing
  3. Incoming inspection
  4. Assembly
  5. Final Inspection
  6. Packaging
  7. Sterilization
  8. Customer Service
  9. Shipping
  10. Management Review
  11. CAPA
  12. Internal Auditing

Why the Process Approach is Recommended

First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments, the process approach will expose it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked.

Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e.,– the element approach). My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are actually incorporated into each process audit.

For example, each process audit requires a review of records as input and outputs. In addition, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment. The tool that BSI uses to teach the process approach is the “Turtle Diagram.” The diagram below illustrates where the name came from.

tutle diagram1 7 Steps to Training Auditors on the Process Approach of the ISO Standard

Process Auditing – “Turtle Diagram”

The Interview

The first skill to teach a new auditor is the interview. Each process audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next, most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique, because it is an “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore, I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.

After getting a general overview, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection, because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.

The next step of this approach is to “determine what resources are used by incoming inspection.” This includes gauges used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list, the auditor should select people to interview and follow-up with a request for training records.

The sixth step  is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures, this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

Process Owners Challenged

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company quality objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review monitoring and measurement of processes, and the trend analysis can be verified to be an input into the CAPA process.

In my brief description of the process approach, I used the incoming inspection process. I typically choose this process for training new auditors, because it is a process that is quite similar in almost every company, and is easy to understand. More importantly, however, the incoming inspection process does an effective job of covering more clauses of the Standard than most audits. Therefore, new auditors get an appreciation for how almost all the clauses can be addressed in one process audit. If you are interested in learning more about Turtle Diagrams and the process approach to auditing, please register for our webinar on the process approach to auditing.

Posted in: ISO Auditing

Leave a Comment (3) →

Auditor Job Responsibilities: The Toughest Thing to Do

The author reveals his thoughts related to auditor job responsibilities and what the toughest thing to do is.

Today was my last day as an external resource for BSI,  and tomorrow will be my last day as an independent consultant. On March 1st, I begin a new job as Sr. Regulatory Affairs Manager for Delcath Systems, Inc. in Queensbury, NY. I am grateful to everyone that I had the pleasure of meeting during the past two-and-half years as a 3rd party auditor, instructor and consultant. I have learned so much from you all. Your parting wishes were very kind and supportive. I sent out emails to as many of you as I could to notify you of this change. Instead of brief acknowledgement and “Good Luck!,”  I received genuine words of thanks and compliments that made me feel very lucky that we have had such an unusual relationship for an auditor and auditee—very much like cats and dogs that learn to live together in the same house.

One of you described the typical relationship with an auditor quite well, “Having an auditor come to your place is always a somehow stressful time. You are always afraid of failing somewhere.” This same person sent me an email last night saying, “I feel like you are one of my friends.” Another auditee walked by me and another team member a few weeks ago while we were waiting for a ride. Instead of avoiding eye contact and walking right on by, he stopped and thanked us for really helping to bring attention to areas that need improvement. This same gentleman had endured a tough interview by me, where I pointed out mistakes in drawings, procedures and his own QC inspection of incoming raw materials. This person has the right attitude.

Auditor Job Responsibilities

As an auditor, we must come to a conclusion as to whether the evidence we collect demonstrates conformity or nonconformity. When we identify nonconformities, we must explain our conclusions. The toughest part of the job is how to “break the news.” If you do it well, the auditee will agree with you and thank you for helping to improve the quality system. If you do it poorly, the auditee will resent you and may even toss you out on your ear.

In my first ever ISO certification audit, I was the auditee and the auditor that interrogated our team was horrible. Not only did the auditor “break the news” poorly, the conclusions were wrong in several instances. To make matters worse, the CEO and the regulatory consultant I had hired were so upset with our auditor that I had to play referee just to keep them from killing the auditor. We received a recommendation for ISO 13485 certification at end of that audit, but I learned a valuable lesson: “Always look at an audit as an opportunity to improve.” The worst that can happen is that the auditor will require you to implement a corrective action. The best that can happen is that you will need to perform internal audits to identify opportunities for corrective actions on your own. Who cares who finds the opportunities to improve?

Auditors and auditees may be cats and dogs, but we should learn to help each other get better without getting upset or feeling anxious.

My third-party auditing days may be done, but I will continue to share my thoughts through this blog, and I hope you will share your feedback too.

Posted in: ISO Auditing

Leave a Comment (2) →

Improving Your ISO Internal Auditing Schedule

 

The author provides tips on how to improve the efficiency and effectiveness of your ISO internal auditing schedule. 

Each week I audit a different company or I teach a group of students how to audit. In the courses I teach, I use a slide that gives an example of an audit schedule (see example below).

%name Improving Your ISO Internal Auditing Schedule

On the surface, this example seems like a good schedule. There are 12 auditors performing two audits each per year. If each auditor spends a day auditing, and another day writing the report, the combined resources equal 48 days (~$20,000) allocated to auditing, and each person spends less than two percent of their work year auditing.

Unfortunately, I have learned that the quality of auditing is directly related to how much time you spend auditing. Therefore, I recommend using fewer auditors. There is no perfect number, but “less is more”. My example also has another fundamental weakness. The audit schedule does not take full advantage of the process approach. Instead of performing an independent audit of document control and training, these two clauses/procedures should be incorporated into every audit. The same is true of maintenance and calibration. Wherever maintenance and calibration is relevant, these clauses should be investigated as part of auditing that area.

For example, when the incoming inspection process is audited, it only makes sense to look for evidence of calibration for any devices used to perform measurements in that area. For a second example…when the production area is being audited, it only makes sense to audit maintenance of production equipment too.

If the concept of process auditing is fully implemented, the following clauses can easily be audited in the regular course of reviewing other processes: 4.2.1), Quality System Documentation, 4.2.3), Document Control, 4.2.4), Record Control, 5.3), Quality Policy, 5.4.1), Quality Objectives, 6.2.2), Training, 6.3), Maintenance, 6.4), Work Environment, 7.1), Planning of Product Realization & Risk Management, 7.6), Calibration, 8.2.3), Monitoring & Measurement of Processes, 8.5.2), Corrective Action, and 8.5.3) Preventive Action. This strategy reduces the number of audits needed by more than half.

Internal Auditing: Upstream/Downstream Examples

Another way to embrace the process approach to auditing is to assign auditors to processes that are upstream or downstream in the product realization process from their own area. For example, Manufacturing can audit Customer Service to better understand how customer requirements are confirmed during the order confirmation process. This is an example of auditing upstream, because Manufacturing receives the orders from Customer Service—often indirectly through an MRP system. Using this approach allows someone from manufacturing to identify opportunities for miscommunication between the two departments. If Regulatory Affairs audits the engineering process, this is an example of auditing downstream. Regulatory Affairs is often defining the requirements for the Technical Files and Design History Files that Engineering creates. If someone from Regulatory Affairs audits these processes, the auditor will realize what aspects of technical documentation are poorly understood by Engineering, and quickly identify retraining opportunities.

One final aspect of the example audit schedule that I think can be improved is the practice of auditing the same process twice per year. This practice doesn’t seem to work very well for a few reasons. First, it requires that an auditor prepare for an audit twice per year and write two reports, instead of one. This doubles the amount of time auditors spend in preparation and follow-up activities associated with an audit. Second, doubling the number of audits naturally shortens the duration of each audit. It is more difficult for auditors to cover all the applicable clauses in a shorter audit, because it takes time to locate records and pursue follow-up trails. Longer audits, covering more clauses, make it easier for the auditor to switch to a different clause while they are waiting for information. Third, if an area is audited every six months, it is often difficult to implement corrective actions and produce evidence of effectiveness before the area is due for auditing again.

It is not possible for me to provide a generic audit schedule that will work for every company, or even to show how all the clauses will be addressed in one table. I can, however, provide an example of an improved schedule that illustrates the above concepts. This example (see below) uses four auditors instead of 12, and the number of days planned for each audit is two days instead of one. The preparation and reporting time is still one day per audit, and therefore the combined resources equal 24 days (~$10,000) allocated to auditing, and each person spends two and one-half percent of their work year auditing.My intention is not to create the perfect plan, but to give audit program managers some new ideas for more efficient utilization of resources. I hope this helps, and please share your own ideas as comments to this posting.

%name Improving Your ISO Internal Auditing Schedule

Posted in: ISO Auditing

Leave a Comment (5) →

Auditing: Effective Training in 7 Steps

Effective auditing requires effective training. Our author, who has extensive experience, provides 7 steps to effectively train new auditors.

Recently, a client asked me to create a training course on how to train operators. I could have taught the operators myself, but there were so many people that needed training, that we felt it would be more cost effective to train the trainers.

Usually, I have multiple presentations archived that I can draw upon, but this time I had nothing. I had never trained engineers on how to be trainers before—at least not formally. I thought about what kinds of problems other quality managers have had in training internal auditors, and how I have helped the auditors improve. The one theme I recognized was that most auditors needed feedback.

Deming Cycle

I finally decided to use the Deming Cycle (Plan-Do-Check-Act, (PDCA) as my framework for the training. Most QA Managers are very experienced and have little trouble planning an audit schedule. The next step is to conduct the audit. The problem is that there is very little objective oversight of the auditing process. The ISO 13485 for medical devices Standard requires that “Auditors shall not audit their own work.” Therefore, most companies will opt for one of two solutions for auditing the internal audit process: 1) hire a consultant, or 2) ask the Director of Regulatory Affairs to audit the internal auditing process.

Both of the above strategies meet the requirements of the ISO 13485 for medical devices Standard, but neither strategy helps to improve  internal auditor’s performance. I have interviewed many audit program managers, maybe 50+, and the most common feedback program managers give is “change the wording of this finding” or “you forgot to close this previous finding.” This type of feedback is related to the report writing phase of the audit process. I rarely hear program managers explain how they help auditors improve at the other parts of the process.

When auditors are first being trained, we typically provide examples of best practices for audit preparation, checklists, interviewing techniques AND reports. After the auditors have been “shadowed” by the program manager for an arbitrary three times, the auditors are now miraculously “trained.” Let’s see if I can draw an analogy that will make my point…

That kind of sounds like watching your 16 year-old drive the family car three times and then giving them a license. I guess that’s why my new Ford Festiva was severely dented on all four sides within six months. You may think my father was a Saint, but I think he might have totaled his tenth car by age 18. At least I contained the damage to one vehicle.

Effective Auditing Requires Effective Training For New Auditors: 7 Steps

The key to training auditors to audit is consistent follow-up over a long period of time (1-2 years depending upon the frequency of audits). I recommend following the same training process that accredited auditors must complete. I have adapted that process and developed seven(7) seven specific recommendations:

1. Have a new auditor observe a few audits before they are allowed to participate (make sure they take notes, and explain what you are doing and why, as you conduct audits they are observing)

2. Have new auditors participate as team members for 10-20 audits, before they are allowed to act as a lead auditor

3. Have new lead auditors conduct team audits with another qualified lead auditor for 10-20 audits before you allow them to conduct an audit alone

4. Shadow new auditors for 100% of their first audit and gradually observe less with each subsequent audit; try to plan the shadowing into your audit agenda

5. Review the notes of new auditors periodically throughout the audit to provide suggestions for improvement and identify missing information

6. Have new lead auditors submit a draft audit agenda to you before submitting it to the supplier or department manager

7. Have new lead auditors rehearse their first few opening and closing meetings with you in private before conducting the opening and closing meeting (make sure they have an opening/closing meeting checklist to help them)

The question is…was my training successful?

Well, how much follow-up training of the trainers did the client ask for?

Posted in: ISO Auditing

Leave a Comment (0) →
Page 2 of 2 12
Follow

Get every new post on this blog delivered to your Inbox.

Join other followers:

Simple Share Buttons
Simple Share Buttons