Blog

Audit schedule and an audit agenda, what’s the difference?

Internal audit and supplier audit programs both require an audit schedule and an audit agenda, but what’s the difference between them?%name Audit schedule and an audit agenda, what’s the difference?

Each week I audit a different company, or I teach a group of students how to audit. In the courses I teach, I use a slide that gives an example of an internal auditing schedule (see the example above). On the surface, this example seems like a good audit schedule. There are 12 auditors performing two audits each year. If each auditor spends a day auditing, and another day writing the report, the combined resources equal 48 days (~$20,000) allocated to auditing, and each person spends less than two percent of their work year auditing.

Unfortunately, I have learned that the quality of auditing is directly related to how much time you spend auditing. Therefore, I recommend using fewer auditors. There is no perfect number, but “less is more.” My example also has another fundamental weakness. The internal auditing schedule does not take full advantage of the process approach to auditing. Instead of performing an independent audit of document control and training, these two clauses/procedures should be incorporated into every audit. The same is true of maintenance and calibration. Wherever maintenance and calibration are relevant, these clauses should be investigated as part of auditing that area.

For example, when the incoming inspection process is audited, it only makes sense to look for evidence of calibration for any devices used to perform measurements in that area. For a second example…when the production area is being audited, it only makes sense to audit maintenance of production equipment too.

If the concept of process auditing is fully implemented, the following ISO 13485 clauses can easily be audited in the regular course of reviewing other processes: 4.2.1), Quality System Documentation, 4.2.3), Document Control, 4.2.4), Record Control, 5.3), Quality Policy, 5.4.1), Quality Objectives, 6.2.2), Training, 6.3), Maintenance, 6.4), Work Environment, 7.1), Planning of Product Realization & Risk Management, 7.6), Calibration, 8.2.3), Monitoring & Measurement of Processes, 8.5.2), Corrective Action, and 8.5.3) Preventive Action. This strategy reduces the number of audits needed by more than half.

Internal Auditing: Upstream/Downstream Examples

Another way to embrace the process approach to auditing is to assign auditors to processes that are upstream or downstream in the product realization process from their own area. For example, Manufacturing can audit Customer Service to understand better how customer requirements are confirmed during the order confirmation process. This is an example of auditing upstream because Manufacturing receives the orders from Customer Service—often indirectly through an MRP system. Using this approach allows someone from Manufacturing to identify opportunities for miscommunication between the two departments. If Regulatory Affairs audits the engineering process, this is an example of auditing downstream. Regulatory Affairs is often defining the requirements for the Technical Files and Design History Files that Engineering creates. If someone from Regulatory Affairs audits these processes, the auditor will realize what aspects of technical documentation are poorly understood by Engineering, and quickly identify retraining opportunities.

One final aspect of the example internal auditing schedule that I think can be improved is the practice of auditing the same process twice per year. This practice doesn’t seem to work very well for a few reasons. First, it requires that an auditor prepare for an audit twice per year and write two reports, instead of one. This doubles the number of time auditors spends in preparation and follow-up activities associated with an audit. Second, increasing the number of audits naturally shortens the duration of each audit. It is more difficult for auditors to cover all the applicable clauses in a shorter audit because it takes time to locate records and pursue follow-up trails. Longer audits, covering more clauses, make it easier for the auditor to switch to a different clause while they are waiting for information. Third, if an area is audited every six months, it is often difficult to implement corrective actions and produce evidence of effectiveness before the area is due for auditing again.

I can’t provide a generic internal auditing schedule that will work for every company or even show how all the clauses will be addressed in one table. I can, however, provide an example of an improved schedule that illustrates the above concepts. This example (see below) uses four auditors instead of 12, and the number of days planned for each audit is two days instead of one. The preparation and reporting time is still one day per audit. Therefore the combined resources equal 24 days (~$10,000) allocated to auditing, and each person spends two and one-half percent of their work year auditing. My intention is not to create the perfect plan, but to give audit program managers some new ideas for more efficient utilization of resources. I hope this helps, and please share your own ideas as comments to this posting.

%name Audit schedule and an audit agenda, what’s the difference?

Posted in: ISO Auditing

Leave a Comment (7) →

Learning Pyramid – 4 Levels of Learning

The author discusses the four levels of learning in the Learning Pyramid, and the lessons learned when he taught an ISO 14971 Risk Management course.%name Learning Pyramid   4 Levels of Learning

I am in Canada, it’s almost midnight, and my client has me thinking so hard that I can’t sleep. I am here to teach the company’s Canadian facility about ISO 14971:2007—the ISO Standard for Risk Management of medical devices.

Most of the companies that request this training are doing so for one of two reasons: 1) several of their design engineers know almost nothing about risk management, or 2) they have several design engineers that are quite knowledgeable concerning risk management, but these engineers have not maintained their credentials, and their last risk management training was related to the 2000 version of the Standard. This company falls into the second category.

I always tell students that I learn something by teaching each course. From this company, however, I have learned so much. This company has forced me to re-read the Standard several times and reflect on the nuances of almost every single phrase. I have learned more about this Standard in one month than I learned in the 3.5 years since I first took the course I am now teaching. 

The four levels of the Learning Pyramid

I have developed a model for learning that explains this phenomenon. I call this model the “Learning Pyramid.” At the base of the pyramid, there are “Newbies.”

This is the first of four levels. At the base, students read policies and procedures with the hope of understanding.

In the second level of the pyramid, the student is now asked to watch someone else demonstrate proper procedures. One of my former colleagues has a saying that explains the purpose of this process well, “A picture tells a thousand words, but a demonstration is like a thousand pictures.” This is what our children call “sharing time,” but everyone over 40 remembers this as “show and tell.”

In the third level of the pyramid, the student is now asked to perform the tasks they are learning. This is described as “doing,” but in my auditing courses, I refer to this process as “shadowing.” Trainees will first read the procedures for Internal Auditing (level 1). Next, trainees will shadow the trainer during an audit as a demonstration of the proper technique (level 2). During subsequent audits, the trainees will audit, and the trainer will shadow the trainee (level 3). During this “doing” phase, the trainer must watch, listen, and wait for what I call the “Teachable Moment.” This is a moment when the trainee makes a mistake, and you can use this mistake as an opportunity to demonstrate a difficult subject.

Finally, in the fourth level of the Learning Pyramid, we now allow the trainee to become a trainer. This is where I am at—so I thought. I am an instructor, but I am still learning. I am learning what I don’t know.

Teaching forces you back to the bottom of the Learning Pyramid

The next step in the learning process is to return to the first level. I am re-reading the Standard and procedures until I understand the nuances that I was unaware of. Then, I will search for examples in the real world that demonstrate these complex concepts I am learning. After searching for examples, I will test my knowledge by attempting to apply the newly acquired knowledge to a 510(k) or CE Marking project for a medical device client. Finally, I will be prepared to teach again.

This reiterative process reminds me of the game Chutes and Ladders, but one key difference is that we never really reach the level of “Guru.” We continue to improve, but never reach our goal of perfection…For further inspiration, try reading “Toyota Under Fire.”

Posted in: Education, ISO 14971:2019 (Risk Management)

Leave a Comment (5) →

Effective Management Skills for Managers

This blog reviews some practical management skills that managers should possess.

Are you frustrated?

Sometimes we hear phrases like: “Well, that’s just an ISO requirement.” This apparent lack of support by top management is what frustrates every Management Representative in the world.

There was a question posted on the Elsmar Cove website on January 10, 2011. In just ten days, there have been 153 postings in response to the original question. As I read through the various postings, I saw several comments about a lack of support from top management.

A little over a decade ago, I was still learning how to supervise people. In an effort to educate myself further, I read a book (sorry I can’t be sure which book anymore). In this book, the boss gave an employee a card with a picture of a baseball bat on it. The instructions provided with this magical card were to use it only when the boss failed to pay attention, and the employee had something important to tell him.

As managers, we assume the impressive title, along with the awesome responsibility. Managers are responsible for leading others. Subordinates are not the “others” I am referring too. The “others” are peers. If you cannot persuade your peers to support you, then you will fail as a manager. The Quality Department cannot fix all the problems. My philosophy is that Quality is responsible for recommending improvements, training people, and helping to implement. We assign corrective actions, but we should be assigning them to the process owner (i.e., – Manager) that is responsible for the area where the problems were created.

Effective Management Skills

If you need help persuading the unenlightened, try picking a project that is critical to the success of the stubborn one. If you can show someone that is currently a detractor how they can apply the Quality principles to help solve their problems, then you will have a convert. Converts become strong supporters. If the stubborn one happens to be at the top, figure out what the CEO’s initiatives are. Initiatives are easy to identify; they talk about it at least twenty times a week. Try showing the CEO how their actions can become Quality Objectives. Show them with graphs. Show up with solutions to their problem. Use the CAPA process as a framework. Show them how the management TEAM can fix it.

If nothing seems to be working, you can always try reviewing some FDA MedWatch reports too–just to scare your boss.

Posted in: ISO Certification

Leave a Comment (0) →

7 Steps to Effective Auditor Training

A five-day lead auditor course is never enough. Effective auditor training must include practical feedback from an experienced auditor.

Recently, a client asked me to create a training course on how to train operators. I could have taught the operators myself, but there were so many people that needed training, that we felt it would be more cost-effective to train the trainers. Usually, I have multiple presentations archived that I can draw upon, but this time I had nothing. I had never trained engineers on how to be trainers before—at least not formally. I thought about what kinds of problems other quality managers have had in training internal auditors, and how I have helped the auditors improve. The one theme I recognized was that effective auditor training needs to include practical feedback from an experienced auditor.

How do you audit the auditing process?

Most quality managers are experienced and have little trouble planning an audit schedule. The next step is to conduct the audit. The problem is that there is very little objective oversight of the auditing process. The ISO 13485 standard for medical devices requires that “Auditors shall not audit their own work.” Therefore, most companies will opt for one of two solutions for auditing the internal audit process: 1) hire a consultant, or 2) ask the Director of Regulatory Affairs to audit the internal auditing process.

Both of the above strategies for auditing the internal audit process meet the requirements of ISO 13485, but neither approach helps to improve an internal auditor’s performance. I have interviewed hundreds of audit program managers over the years, and the most common feedback audit program managers give is “change the wording of this finding” or “you forgot to close this previous finding.” This type of feedback is related to the report writing phase of the audit process. I rarely hear program managers explain how they help auditors improve at the other parts of the process.

When auditors are first being trained, we typically provide examples of best practices for audit preparation, checklists, interviewing techniques, AND reports. After the auditors have been “shadowed” by the program manager for an arbitrary three times, the auditors are now miraculously “trained.” Let’s see if I can draw an analogy that will make my point…

That kind of sounds like watching your 16-year-old drive the family car three times and then giving them a license. I guess that’s why my new Ford Festiva was severely dented on all four sides within six months. You may think my father was a Saint, but I think he might have totaled his tenth car by age 18. At least I contained the damage to one vehicle.

7 Steps to Effective Auditor Training

The key to training auditors to audit is consistent follow-up over a long period of time (1-2 years depending upon the frequency of audits). I recommend following the same training process that accredited auditors must complete. I have adapted that process and developed seven (7) specific recommendations:

  1. Have a new auditor observe a few audits before they are allowed to participate (make sure they take notes and explain what you are doing and why, as you conduct audits they are observing)
  2. Have new auditors join as team members for 10-20 audits, before they are allowed to act as a lead auditor
  3. Have new lead auditors conduct team audits with another qualified lead auditor for 10-20 audits before you allow them to conduct an audit alone
  4. Shadow new auditors for 100% of their first audit and gradually observe less with each subsequent audit; try to plan the shadowing into your audit agenda
  5. Review the notes of new auditors periodically throughout the audit to provide suggestions for improvement and identify missing information
  6. Have new lead auditors submit a draft audit agenda to you before sending it to the supplier or department manager
  7. Have new lead auditors rehearse their first few opening and closing meetings with you in private before conducting the opening and closing meeting (make sure they have an opening/closing meeting checklist to help them)

About the Author

Rob Packard 150x150 7 Steps to Effective Auditor TrainingRobert Packard is a regulatory consultant with 25+ years of experience in the medical device, pharmaceutical, and biotechnology industries. He is a graduate of UConn in Chemical Engineering. Robert was a senior manager at several medical device companies—including the President/CEO of a laparoscopic imaging company. His Quality Management System expertise covers all aspects of developing, training, implementing, and maintaining ISO 13485 and ISO 14971 certification. From 2009-2012, he was a lead auditor and instructor for one of the largest Notified Bodies. Robert’s specialty is regulatory submissions for high-risk medical devices, such as implants and drug/device combination products for CE marking applications, Canadian medical device applications, and 510(k) submissions. The most favorite part of his job is training others. He can be reached via phone 802.258.1881 or email. You can also follow him on Google+LinkedIn or Twitter.

Posted in: ISO Auditing

Leave a Comment (0) →
Page 30 of 30 «...10202627282930