Blog

Archive for ISO Auditing

Internal Audit Training for New Hires

 

welcome aboard Internal Audit Training for New Hires

The author discusses a few proven internal audit training strategies (i.e., shadowing, auditing process owners) for new hires.

Once you have identified someone that you want to “hire” as an internal auditor, your next step should be to develop an “Onboarding plan for them with their boss. If you are hiring someone that will be a dedicated auditor, please ignore my quotation marks above. In most companies, however, the internal auditors are volunteers that report to another hiring manager. Therefore, as the audit program manager, you need to get a firm commitment from the auditor’s boss with regard to the time required to train the new auditor and to perform audits on an ongoing basis. 

Winning Over the Boss

In my previous posting, I said that “The biggest reason why you want to be an auditor is that it will make you more valuable to the company.” The auditor’s boss may or may not agree with this statement, but the boss knows that the salary is coming out of their budget either way. Therefore, talk with the auditor’s boss and determine what the auditor’s strengths and weaknesses are. Find out which skills the boss would like to see the auditor develop. By doing this, the two of you can develop a plan for making the auditor more valuable to their boss AND the company. 

Making Re-Introductions

Ideally, auditors are extraverted and have worked at the company long enough to know the processes and process owners that they will be assigned to audit—especially if they will be auditing upstream and downstream from their process area. In the past, the auditor may have been a customer or a supplier, but now the relationship with a process owner will change. Auditors are required to interview process owners, and this involves asking tough questions that might not be appropriate in the auditor’s regular job duties. Therefore, as the audit program manager, you should re-introduce the auditor to the process owner in their new capacity as an auditor. During this re-introduction, it is important to make three points:

  1. The auditor is going to be trained first
  2. You will be shadowing the auditor during the audit, and
  3. The auditor’s job is to help the process owner identify opportunities for improvement

By making the first point, you are reminding the process owner of the scheduled audit—well in advance. You are also informing the process owner that this auditor will have new skills, and the process owner should have some tolerance for mistakes that new employees make. You might also mention that you would like to get the process owner’s feedback after the audit, so the auditor knows which areas they need to improve upon to become better auditors. The second point should put the process owner at ease—assuming the process owner has a good relationship with you as the audit program manager. It is important to be descriptive when “shadowing” is mentioned. Both the process owner and the auditor may not understand the process or the purpose of shadowing. The following blog posting might help with this: “How do you shadow an auditor? Did you learn anything?”

The third point is the most critical step in onboarding a new auditor. For an auditor to be successful, they must ADD VALUE! As an auditor, you cannot pretend to add value. The process owner should know their process, and they probably know which areas are weakest. The audit program manager should encourage the process owner to list some specific areas in which they are having problems. Ideally, the process owner would be informed of this need before the re-introduction. Then the process owner can be better prepared for the meeting, and hopefully, they will have a few target areas already identified. Targets with associated metrics are the best choice for a new auditor because these targets reinforce the process approach to auditing. 

Next Steps for Internal Audit Training

Once your new auditor has been re-introduced to the process owners, they will be auditing, and you need to begin the training process. As with any new employee, it is important to document training requirements and to assess the auditor’s qualifications against the requirements of an auditor. Every new auditor will need some training, but the training should be tailored specifically to the needs of the auditor. The training plan for a new auditor should include the following:

  1. A reading list of company procedures specific to auditing and external standards that are relevant
  2. Scheduled dates for the auditor to shadow another experienced auditor
  3. Scheduled dates for an experienced auditor to shadow the auditor during the first two process audits (upstream and downstream)
  4. Goals and objectives for the internal audit program; and
  5. Any training goals that the auditor’s boss has identified for the auditor

 

Posted in: ISO Auditing

Leave a Comment (0) →

Effective Recruiting for the Auditor Position

help wanted Effective Recruiting for the Auditor Position

Stop begging people to help you audit. Learn how to recruit auditors more effectively.

This blog shares thoughts related to effectively recruiting for the auditor position. One suggestion may surprise you.

Nearly 100% of the people I train as auditors were not hired specifically to be auditors. Instead, auditing is something extra that they were asked to do, in addition to their regular job. This situation creates three problems for the audit program manager:

  1. You have difficulty getting enough people to perform the audits.
  2. Most auditors will come from your department, so who is going to audit you?
  3. Auditors have little or no motivation to develop their auditing skills.

Stop begging for “volunteers” from other departments and start recruiting.

When I am recruiting someone to audit, I always get asked two questions:

  1. Who/What will I be auditing?
  2. What will I have to do?

You need to motivate people to become auditors because it requires extra work. The answer to #2 should be specific. I recommend creating a “sell sheet” that explains the process of performing an audit. I also like to develop educational sell sheets. Therefore, I recommend adopting the flow chart in ISO 19011:2011 (Figure 2 on page 15). I would add time estimates for each step of the process (6.2 – 6.7). This will serve as a training tool for future auditors and will eliminate the fear of an unknown time commitment for your potential recruit.

In order to answer #1, I recommend you assign the recruit processes that are upstream and downstream. I have recommended this concept in previous postings, but essentially you are assigning the person to audits of internal suppliers and internal customers. By doing this, utilizing the process approach will be more natural to the auditor and they will have a vested interest in doing a thorough audit. This also creates a situation where the auditor is typically assigned to at least two process audits per year.

The next question is one that your potential recruit will never ask, but they are always thinking…

Why should I become an auditor?

The biggest reason why you want to be an auditor is that it will make you more valuable to the company.

Auditors are required to interview department managers and ask tough questions. This gives the auditor a better understanding of the organization as a whole, and it gives them insight into how other managers work. This insight is pure gold.

If you want to be effective and get promoted, you need to demonstrate value to your boss and top management. If you don’t understand what other departments need, how can you help them? No manager will promote a selfish, power-hungry hog. They promote team players that make others better. Auditing gives you the insight necessary to understand how you can do that.

Auditing other departments will also give you insider information as to where new job openings will be. Sometimes you can’t wait for your boss to get promoted. In that case, you might want to know more about other departments in your company.

Each corporate culture is different, but the audit program manager needs to “sell” the recruit on volunteering to be an auditor.

Where to find recruits

Due to the cross-functional nature of auditing, I have found that my own personal experience working in multiple departments was invaluable. I have a better understanding of how a department functions than other auditors because I have worked in that department at another company. Operations, engineering and research experience are extremely valuable for auditing, but I believe the experience that transfers best to auditing is any position where you are addressing customer complaints and returns—such as technical support or service.

If your company is large enough to hire full-time auditors, I recommend searching for potential auditors at your suppliers and their competitors. These people will bring unique knowledge that is critical to a successful supplier selection process, and these individuals will increase the diversity in your company—instead of duplicating knowledge and expertise.

Posted in: ISO Auditing

Leave a Comment (0) →

How to Finish your Audit Schedule by December 31st

This blog provides viable options to consider related to successfully completing your audit schedule by year’s end.

Let’s say that there are 34 days until the end of 2012. You have four supplier audits and three internal audits to complete. Of course, all but two of these audits are overdue. What should you do?

Options that might be readily available to you include:

  1. Get some help
  2. Perform remote audits
  3. Reschedule some of the audits for next year

There are some great cartoons and jokes about doing more with less, but if you intend to complete seven audits before the end of the year, you might need some help. There really isn’t any time left to train someone, so that they are capable of conducting an effective audit by themselves. I expect to prepare a new auditor to take at least six months before I believe they are ready to work solo. Even if you are less demanding than I am, you still would need time for classroom training and shadowing a couple of audits. Therefore, the best I believe you could hope for is one or two solo audits of the seven you need to complete.

Realistically, your only source of help would be auditors that are already trained and consultants. The last month of the year is historically hectic for everyone–especially quality assurance auditors. Therefore, consultants will not be cheap, and you should commit to any qualified consultants that are available without too much delay (then again, maybe they are available because they are not very good). If you have any in-house auditors that are already trained, do everything you can to get some of their time in the next few weeks.

Remote Audits

Option two is to perform remote audits. This is a viable option for you to justify for a supplier with an impressive quality track record, or suppliers in other countries. However, a remote audit is not the same as asking a supplier to complete a survey. ISO 19011:2011 provides some guidance specific to remote auditing in table B.1 of Annex B.

For a remote audit, you should still sample just as many records—if not more. You should conduct interviews by phone, Skype, or some similar technology. You should analyze any available data to help identify which processes appear to be effective and which processes need to improve. If you are performing a remote audit for the first time, I recommend focusing on the same processes that you would normally audit in a conference room, rather than processes that you would typically audit where they occur—such as production controls. Regardless of which process you check, you should always request data.

Option three is to reschedule some audits for January 2013. I have suggested this so many times to clients, but very few follow this advice. If your company is late in conducting some audits, the important thing to do is to document this, reschedule the audits, and take corrective action(s) to prevent it from recurrence. If you wait until January, you will have additional time to train an auditor, as well. Finally, consultants historically have more time available in January than December.

In parallel with your efforts to catch-up on your schedule, I also recommend the following:

Create a quality objective that measures the “on-time delivery” of audits and audit reports. This is an effective metric for managing an audit program.

Investigate the reasons for audits being overdue. If the occurrence was preventable, then I recommend initiating a CAPA. This will have two effects. First, your third-party auditors will see that you have identified the problem yourself and taken appropriate corrective action(s). If you also discuss this during a Management Review, this information can be used effectively to change the grading of an audit finding to a “minor,” or to potentially eliminate the finding altogether. Second, it will ensure that this doesn’t occur again.

Posted in: ISO Auditing

Leave a Comment (0) →

“Shadowing” as an Effective Auditor Quality Training Technique

%name Shadowing as an Effective Auditor Quality Training Technique

If you are shadowing, you are taking notes, so you can discuss your observations with the person you are shadowing later.

This article reviews shadowing as an effective auditor quality training technique in a first, second, and third-party audit. Also, five common shadowing mistakes trainers make is discussed as well.

Somewhere in your procedure for “Quality Audits,” I’ll bet there is a section on auditor competency. Most companies require that the auditor has completed either a course for an internal auditor or a lead auditor course. If the course had an exam, then you might even have evidence for training effectiveness. Demonstrating competency is much harder. One way is to review internal audit reports, but writing reports are just part of what an auditor does. How can you evaluate an auditor’s ability to interview people, take notes, follow audit trails, and manage their time? The most common solution is to require that the auditor “shadow” a more experienced auditor several times, and then the trainee will be “shadowed” by the trainer. 

Shadowing 1st Party Audits

ISO 19011:2011 defines first-party audits as internal audits. When first-party auditors are being shadowed by a trainer or vice versa, there are many opportunities for training. The key to the successful training of auditors is to recognize teachable moments.

When the trainer is auditing, the trainer should look for opportunities to ask the trainee, “What should I do now?” or “What information do I need to record?” In these situations, the trainer is asking the trainee what they should do BEFORE they do it. If the trainee is unsure, the trainer should explain what, why, and how at that moment with real examples.

When the trainer is shadowing, the trainer should watch and wait for a missed opportunity to gather important information. In these situations, the trainer must resist guiding the trainee until after the trainee appears to be done. When it happens, sometimes the best tool is simply asking, “Are you sure you got all the information you came for?”

Here are five (5) mistakes that I had observed trainers make when they were shadowing:

1. Splitting up, instead of staying together, is one of the more common mistakes I have observed. This happens when people are more interested in completing an audit, rather than taking every advantage of training opportunities. The trainee may be capable of auditing on their own, but this is unfair to the trainee because they need feedback on their auditing technique. This is also unfair to the auditee because it is challenging to support multiple auditors simultaneously. When it is unplanned, there may not be trainers available for both auditors. If an audit is running behind schedule, this is the perfect time to teach a trainee how to recover sometime in their schedule. Time management is, after all, one of the hardest skills for auditors to master.

2. Staying in the conference room, instead of going to where the work is done, is a common criticism of auditors. If the information you need to audit can be found in a conference room, then you could have completed the audit remotely. This type of audit teaches new auditors very little, other than how to take notes. These are necessary skills that auditors should master in a classroom before shadowing.

3. Choosing an administrative process is a mistake because administrative processes limit the number of aspects of the process approach that can be practiced by an auditor-in-training. Administrative processes rarely have equipment that requires validation or calibration, and both the process inputs and outputs consist only of paperwork, forms, or computer records. With raw materials and finished goods to process, the job of the auditor is more challenging, because there is more to be aware of.

4. Not providing honest feedback is a huge mistake. Auditors need to be thick-skinned, or they don’t belong in a role where they are going to criticize others. Before you begin telling other people how to improve, you first need to self-reflect and identify your own strengths and weaknesses. Understanding your perspective, strengths, weaknesses, and prejudices is critical to being an effective assessor. As a trainer, it is your job to help new auditors to self-reflect and accurately rate their performance against objective standards.

5. “Silent Shadowing” has no value at all. By this, I mean shadowing another auditor without asking questions. If you are a trainee, you should be mentally pretending you are doing the audit. Whenever the trainer does something different from the way you would do things, you should make a note so you can ask, “Why did you do that?” If you are the trainer, you should also be mentally pretending you are doing the audit. It is not enough to be present. Your job is to identify opportunities for the trainee to improve. The better the trainee, the more challenging it becomes to identify areas for improvement.  This is why training other auditors have helped me improve my auditing skills.

Shadowing Second-Party Audits

supply chain weakest link Shadowing as an Effective Auditor Quality Training Technique

If you are developing a new supplier quality engineer that is responsible for performing supplier audits, it is recommended to observe the auditor during some actual supplier audits. Supplier audits are defined as second-party audits in the ISO 19011:2011 Standard. The purpose of these audits is not to verify conformity to all the aspects of ISO 13485. Instead, the primary purpose of these audits is to verify that the supplier has adequate controls in place to consistently manufacture conforming products for your company. Therefore, processes such as Management Review (Clause 5.6) and Internal Auditing (Clause 8.2.2) are not typically sampled during a second-party audit.

The two most valuable processes for a second-party auditor to sample are 1) incoming inspection, and 2) production controls. Using the process approach to auditing, the second-party auditor will have an opportunity to verify that the supplier has adequate controls for documents and records for both of these processes. Training records for personnel performing these activities can be sampled. The adequacy of raw material storage can be evaluated by following the flow of accepted raw materials, leaving the incoming inspection area. Calibration records can be sampled by gathering equipment numbers from calibrated equipment in use by both processes. Even process validation procedures can be assessed by comparing the actual process parameters being used in manufacturing with the documented process parameters in the most recent validation or re-validation reports.

My recommendation is to have the trainee shadow the trainer during the process audit of the incoming inspection process, and for the trainer to shadow the trainee during the process audit of production processes. In between the two process audits, the trainee should be asking questions to help them fully understand the process approach to auditing. Supplier auditors should also be coached on techniques for overcoming resistance to observe processes that may involve trade secrets, or where competitor products may also be present. During the audit of production processes, the trainer may periodically prompt the trainee to gather the information that will be needed for following audit trails to calibration records, document control, or for comparison with the validated process parameters. The “teachable moment” is immediately after the trainee missed an opportunity, but while the trainee is still close enough to go back and capture the missing details.

Shadowing Third-Party Audits

qsit inspection Shadowing as an Effective Auditor Quality Training Technique

Use your FDA inspections and ISO certification audits as an opportunity to shadow experienced auditors and to learn what they are looking for.

If you are going to shadow a third-party auditor, I recommend two specific people to “shadow” the auditor. First, the process owner should be the guide for whichever process is being audited. This is the person that will be responsible for addressing any nonconformities found in the area, and they should be present during interviews–although they should be coached on when to comment and when to remain quiet and simply observe.  Second, the person that performed an internal audit of the process being audited should be present if at all possible. This person will benefit from seeing how a professional third-party auditor performs a process audit because they will know which things to look for in the future so that auditees in that area are prepared for the next external audit.

For other sources of information related to auditor shadowing, please check out the following links:

1. Internal Auditor Training – Shadowing external auditor? – from Elsmar Cove

2. Developing Supplier Quality Auditor Training Programs – by Seth Mailhot at Nixon Peabody

Posted in: ISO Auditing

Leave a Comment (0) →

Instructor Effectiveness and the Power of a SNICKERS

The author discusses his personal experience attending a training class, instructor effectiveness, and reasons why he learned so much there.

I guess there are still some instructors out there that need to be reminded that we can all read the regulations on our own. We don’t need to pay $1,000+ per day to have someone read stuff for us. If that’s what you want, my 10-year old son is a fantastic reader. He’ll record anything you want, in any media format, for a much smaller dollar figure. If you want to learn something that is worth at least as much as your investment of time and money, then you need to find an instructor that can teach effectively.

Four Prerequisites for a Great Instructor:

1. The instructor must be an expert

2. The instructor must inspire participation

3. The instructor must provide practical examples for each student

4. The instructor must get everyone’s attention–and keep it

The most important determining factor of training effectiveness, however, occurs after the course is over When you are teaching quality assurance and regulatory affairs, you must develop your ability to inspire and engage students to Olympic medalist proportions. “Blah, blah, blah…” and “Death by PowerPoint” will get you fired. Don’t read your slides, don’t turn your back on the audience (or they’ll attack) and PLEASE don’t ever ask someone to read the definition of nonconformity out loud to the rest of the group. When I teach a class, you demand my best. I’m six-foot, six inches tall, and I have a loud booming voice. My mother has red hair, and she was an opera singer. I’ve got the voice to fill any auditorium and stage presence to match. But if you even start to nod off in class, I may just have to throw a Snickers bar at you.

snickers Instructor Effectiveness and the Power of a SNICKERS

This is an essential tool for any instructor. It functions as a tool to prod sleeping students awake, is small enough to cause minimal injury when thrown, serves as an emergency food supply, and is gluten-free.

If legal counsel recommends against using projectiles to encourage class participation, you might also consider one of my all-time genius ideas. I was scheduled for a two-day course in Ottawa, but the day before, I needed to perform an audit in Pennsylvania. Therefore, my flight was the last flight into Ottawa–arriving at approximately 1 a.m. My flight was delayed for more than an hour, and the person in front of me was trying to smuggle an extra carton of smokes into the country. Just before 4 a.m., my taxi arrived at the Albert at Bay Suite Hotel. The class started at 8 a.m. I made it to class on time, and excessive consumption of several pots of black coffee helped get me to lunch. Then my legs started getting a little shaky. Fortunately, there was a convenience store next door that sold my favorite chocolate–the Dark Aero bar! After four of these monstrous doses of cacao, and another pot of coffee, I could have listened to the lecture on the Canadian Medical Device Regulations all night.

aero bar Instructor Effectiveness and the Power of a SNICKERS

Hershey’s copied them, but the result was a mere shadow of Nestle’s greatness. Canadians know how to make junk food, tell a joke, and play hockey!

Lessons Learned

Despite the physical handicap of sleep deprivation, I still learned a ton from my course in Canada. Here’s why:

1. The instructors were experts. Both instructors were regulatory experts and Canadian. Both instructors taught this course twice a year for multiple years, and one of the instructors actually worked for Health Canada.

2. The instructors were blessed with the perfect audience that was hyper-motivated to pass the course. Everyone in the class worked for a Notified Body that had sponsored them to take the course. In order to stay employed and get a raise, I needed to pass that course. If I failed the exam, I had to absorb the cost to travel back to Ottawa and retake the course in February (BRRRR!).

3. Everyone has different experiences, and therefore not every example makes sense to us. Therefore, instructors need to use practical examples that are actionable. In this course, the instructors brought more than a dozen medical devices to the class. We studied the labeling and intended use of each device. Even students from Japan, Europe, and Australia were familiar with some of the products. This was critical because we all needed to be able to identify incorrect Canadian labeling.

4. The greatest asset of all was the humor of the instructor from Health Canada. He was hilarious. He had everyone laughing at his jokes for the entire course. Most of the jokes were not funny enough for a stand-up routine, but this was a mandatory regulatory course on Canadian regulations. Who would even expect a chuckle? Despite the strengths of these instructors, there is only one reason why I know the Canadian Medical Device Regulations (CMDR), as well as I do. I use them every single week.

Some Examples of How I Used the CMDR:

First, I had to audit 162 days for BSI in 2011. Ninety percent of those 162 days were for companies that required a Canadian Medical Device License. Therefore, I started auditing companies to the Canadian regulations immediately after the course. Second, I was also consulting for companies at the same time I was auditing for BSI. Consulting clients hired me to prepare and submit the Canadian Medical Device License Applications for them. I also had to revise and create new procedures specific to Canadian regulations. I spent another 60+ days in 2011 doing consulting. Finally, I was one of BSI’s instructors that taught the regulatory comparison course, which compared the regulations of the USA, Canada, Europe, Australia, and Japan.

Therefore, at least once a month, I had a classroom of 6-20 people asking me challenging questions about how to interpret and apply regulations from each of these countries to their products. I used every bit of knowledge I learned in that course in Ottawa, and I started using that knowledge immediately after the course. I had peers, superiors, clients, and students challenging my knowledge of these topics every day. This is what makes you a subject matter expert. If you need to learn something about Quality Assurance or Regulatory Affairs, a one-hour webinar, reading a blog, taking a five-day, or shadowing another more experienced person is not enough. In the end, all of the above will get you to the level of barely competent!  If you want to learn, you need a great instructor. Then you need to use everything you learned at every opportunity for several years. Some say, “If you can’t do, teach.” I say, “Bring a SNICKERS bar and throw it at them for faking it.”

Posted in: ISO Auditing

Leave a Comment (0) →

Auditing Design Controls – 7 Step Process

This blog reviews seven steps for effectively auditing design controls utilizing the ISO 13485 standard and process approach to auditing.

turtle diagram for design controls Auditing Design Controls   7 Step Process

Third-party auditors (i.e., – a Notified Body Auditor) don’t always practice what we preach. I know this may come as a huge shock to everyone, but sometimes we don’t use the process approach. Auditing design controls is a good example of my own failure to follow was it true and pure. Instead, I use NB-MED 2.5.1/rec 5 as a checklist, and I sample Technical Files to identify any weaknesses. The reason I do this is that I want to provide as much value to the auditing client as possible without falling behind in my audit schedule.

Often, I would sample a new Technical File for a new product family that had not been sampled by the Technical Reviewer yet. My reason for doing this is that I could often find elements that are missing from the Technical File before the Technical Reviewer saw the file. This gives the client an opportunity to fix the deficiency before submission and potentially shortens the approval process. Since NB-MED documents are guidance documents, I could not write the client up for a nonconformity, unless they were missing a required element of the M5 version of the MDD (93/42/EEC as modified by 2007/47/EC). This is skirting the edge of consulting for a third- party reviewer, but I found it was a 100% objective way to review Technical Files. I also found I could review an entire Technical File in about an hour.

What’s wrong with this approach to auditing design controls?

This approach only tells you if the elements of a Technical File are present, but it doesn’t evaluate the design process. Therefore, I supplemented my element approach with a process audit of the design change process by picking a few recent design changes that I felt were high-risk issues. During the process audit of the design change process, I sampled the review of risk management documentation, any associated process validation documentation, and the actual design change approval records. If I had time, I looked for the following types of changes: 1) vendor change, 2) specification change, and 3) process change. By doing this, I covered the following clauses in ISO 13485:2016: 7.4 (purchasing), 7.3.9 (design changes), 7.5.6 (process validation), 7.1 (risk management), and 4.2.5 (control of records).

So what is my bastardized process approach to auditing design controls missing? Clauses 7.3.1 through 7.3.10 of ISO 13485:2016 are missing. These clauses are the core of the design and development process. To address this, I would like to suggest the following process approach:

Step 1 – Define the Design Process

Identify the process owner and interview them. Do this in their office–not in the conference room. Get your answers for steps 2-7 directly from them. Ask lots of open-ended questions to prevent “yes/no” responses.

Step 2 – Process Inputs

Identify how design projects are initiated. Look for a record of a meeting where various design projects were vetted and approved for internal funding. These are inputs into the design process. There should be evidence of customer focus, and some examples of corrective actions taken based upon complaints or service trend analysis.

Step 3 – Process Outputs

Identify where Design History Files (DHF) are stored physically or electronically, and determine how the DHF is updated as the design projects progress.

Step 4 – What Resources

This is typically the step of a process audit where their auditor needs to identify “what resources” are used in the process. However, only companies that have software systems for design controls have resources dedicated to Design and Development. I have indicated this in the “Turtle Diagram” presented above.

Step 5 – With Whom, Auditing Training Records

Identify which people are assigned to the design team for a design project. Sometimes companies assign great teams. In this case, the auditor should focus on the team members that must review and approve design inputs (see Clause 7.3.2) and design outputs (see Clause 7.3.3). All of these team members should have training records for Design Control procedures and Risk Management procedures.

Step 6 – Auditing Design Controls Procedures and Forms

Identify the design control procedures and forms. Do not read and review these procedures. Auditors never have the time to do this. Instead, ask the process owner to identify specific procedures or clauses within procedures where clauses in the ISO Standard are addressed. If the process owner knows exactly where to find what you are looking for, they’re training was effective, or they may have written the procedure(s). If the process owner has trouble locating the clauses you are requesting, spend more time sampling training records.

Step 7 – Process Metrics

Ask the process owner to identify some metrics or quality objectives they are using to monitor and improve the design and development process. This is a struggle for many process owners–not just design. If any metrics are not performing up to expectations, there should be evidence of actions being taken to address this. If no metrics are being tracked by the process owner, you might review schedule compliance.

Many design projects are behind schedule, and therefore this is an important metric for most companies. Now that you have completed your “Turtle Diagram,” if you have more time to audit the design process, you can interview team members to review their role in the design process. You could also sample-specific Technical Files as I indicated above. If you are performing a thorough internal audit, I recommend doing both. To learn more about using the process approach to auditing, you can register for our webinar on the topic.

Posted in: Design Control, ISO Auditing

Leave a Comment (1) →

Auditing Medical Device Software Vendors

This blog presents some thoughts related to auditing medical device software companies.

Software medical devices are used to assist medical professionals. For example, radiologists use software with identifying areas of interest for medical imaging. Do you know how to audit a software company?

As a third-party auditor, I have had the pleasure of auditing software companies for CE Marking. When you audit a software company for the first time, this forces you to re-learn the entire ISO 13485 Standard. For example, if a company only produces software, there is very little to sample for incoming inspection and purchasing records. This is because the product is not physical—it’s software. Clauses of ISO 13485 related to sterility, implants, and servicing are also not applicable to software products. If the software is web-based, the shipping and distribution clauses (i.e., – 7.5.5) might present a challenge to an auditor as well.

The aspects of the ISO 13485 Standard that I found to be the most important to auditing software products were design controls and customer communication. Many auditors are trained in auditing the design and development of software, but very few auditors have experience auditing technical support call centers. When auditing a call center, most of the calls represent potential complaints related to software “bugs,” system incompatibilities with the operating system or hardware, and use errors resulting from the design of the user interface.

In most technical support call centers, the support person tries to find a work-around for problems that are identified. The problem with a “work-around” is that it is the opposite approach to the CAPA process. To meet ISO 13485 requirements, software companies must show evidence of monitoring and measuring these “bugs.” There must also be evidence of management identifying negative trends and implementing corrective actions when appropriate.

As an auditor, you should focus on how the company prioritizes “bugs” for corrective actions. Most software companies focus on the severity of software operations and the probability of occurrence. This is the wrong approach. Failure to operate is not the most severe result of medical device software failure. Medical device software can result in injury or death to patients. Therefore, it is critical to use a risk-based approach to prioritization of CAPAs. This risk-based approach should focus on the severity of effects upon patients—not users. This focus on safety and efficacy is an essential requirement of the Medical Device Directive (93/42/EEC as modified by 2007/47/EC) and is a requirement of ISO 14971:2007.

Posted in: ISO Auditing

Leave a Comment (0) →

Process Approach to Auditing – 7 Steps to Training Auditors

The process approach to auditing is demonstrated using Turtle Diagrams as a tool instead of using traditional auditor checklists.

tutle diagram1 Process Approach to Auditing   7 Steps to Training Auditors

I have been reviewing trends for how people find my website, and a large number of you appear to be interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.

First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead, you audit each process. Typical processes include:
  1. Design & Development
  2. Purchasing
  3. Incoming inspection
  4. Assembly
  5. Final Inspection
  6. Packaging
  7. Sterilization
  8. Customer Service
  9. Shipping
  10. Management Review
  11. CAPA
  12. Internal Auditing

Why the Process Approach is Recommended

First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments, the process approach will expose it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked.

Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e.,– the element approach). My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are incorporated into each process audit.

For example, each process audit requires a review of records as input and outputs. Also, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment. The tool that BSI uses to teach the process approach is the “Turtle Diagram.” The diagram above illustrates where the name came from.

Interviewing with the Process Approach

The first skill to teach a new auditor is the interview. Each process approach audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next, most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique because it is a “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore, I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.

After getting a general overview, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.

The next step of this approach is to “determine what resources are used by incoming inspection.” This includes gauges used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list, the auditor should select people to interview and follow-up with a request for training records.

The sixth step is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures, this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

Challenging Process Owners

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company quality objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review the monitoring and measurement of processes, and the trend analysis can be verified to be input into the CAPA process.

In my brief description of the process approach, I used the incoming inspection process. I typically choose this process for training new auditors because it is a process that is quite similar in almost every company, and it is easy to understand. More importantly, however, the incoming inspection process does an effective job of covering more clauses of the Standard than most audits. Therefore, new auditors get an appreciation for how almost all the clauses can be addressed in one process audit. If you are interested in learning more about Turtle Diagrams and the process approach to auditing, please register for our webinar on the process approach to auditing.

Posted in: ISO Auditing

Leave a Comment (3) →

Auditor Job Responsibilities: The Toughest Thing to Do

The author reveals his thoughts related to auditor job responsibilities and what the toughest thing to do is.

Today was my last day as an external resource for BSI,  and tomorrow will be my last day as an independent consultant. On March 1st, I begin a new job as Sr. Regulatory Affairs Manager for Delcath Systems, Inc. in Queensbury, NY. I am grateful to everyone that I had the pleasure of meeting during the past two-and-half years as a 3rd party auditor, instructor, and consultant. I have learned so much from you all. Your parting wishes were very kind and supportive. I sent out emails to as many of you as I could to notify you of this change. Instead of brief acknowledgment and “Good Luck!” I received genuine words of thanks and compliments that made me feel very lucky that we have had such an unusual relationship for an auditor and auditee—very much like cats and dogs that learn to live together in the same house.

One of you described the typical relationship with an auditor quite well, “Having an auditor come to your place is always a somehow stressful time. You are always afraid of failing somewhere.” This same person sent me an email last night saying, “I feel like you are one of my friends.” Another auditee walked by another team member and me a few weeks ago while we were waiting for a ride. Instead of avoiding eye contact and walking right on by, he stopped and thanked us for really helping to bring attention to areas that need improvement. This same gentleman had endured a tough interview by me, where I pointed out mistakes in drawings, procedures, and his own QC inspection of incoming raw materials. This person has the right attitude.

Auditor Job Responsibilities

As an auditor, we must come to a conclusion as to whether the evidence we collect demonstrates conformity or nonconformity. When we identify nonconformities, we must explain our findings. The toughest part of the job is how to “break the news.” If you do it well, the auditee will agree with you and thank you for helping to improve the quality system. If you do it poorly, the auditee will resent you and may even toss you out on your ear.

In my first-ever ISO certification audit, I was the auditee, and the auditor that interrogated our team was horrible. Not only did the auditor “break the news” poorly, but the conclusions were also wrong in several instances. To make matters worse, the CEO and the regulatory consultant I had hired were so upset with our auditor that I had to play referee just to keep them from killing the auditor. We received a recommendation for ISO 13485 certification at the end of that audit, but I learned a valuable lesson: “Always look at an audit as an opportunity to improve.” The worst that can happen is that the auditor will require you to implement corrective action. The best that can happen is that you will need to perform internal audits to identify opportunities for corrective actions on your own. Who cares who finds the opportunities to improve?

Auditors and auditees maybe cats and dogs, but we should learn to help each other get better without getting upset or feeling anxious.

My third-party auditing days may be done, but I will continue to share my thoughts through this blog, and I hope you will share your feedback too.

Posted in: ISO Auditing

Leave a Comment (2) →

Improving Your ISO Internal Auditing Schedule

 

The author provides tips on how to improve the efficiency and effectiveness of your ISO internal auditing schedule. 

Each week I audit a different company, or I teach a group of students how to audit. In the courses I teach, I use a slide that gives an example of an audit schedule (see example below).

%name Improving Your ISO Internal Auditing Schedule

On the surface, this example seems like a good schedule. There are 12 auditors performing two audits each year. If each auditor spends a day auditing, and another day writing the report, the combined resources equal 48 days (~$20,000) allocated to auditing, and each person spends less than two percent of their work year auditing.

Unfortunately, I have learned that the quality of auditing is directly related to how much time you spend auditing. Therefore, I recommend using fewer auditors. There is no perfect number, but “less is more.” My example also has another fundamental weakness. The audit schedule does not take full advantage of the process approach. Instead of performing an independent audit of document control and training, these two clauses/procedures should be incorporated into every audit. The same is true of maintenance and calibration. Wherever maintenance and calibration are relevant, these clauses should be investigated as part of auditing that area.

For example, when the incoming inspection process is audited, it only makes sense to look for evidence of calibration for any devices used to perform measurements in that area. For a second example…when the production area is being audited, it only makes sense to audit maintenance of production equipment too.

If the concept of process auditing is fully implemented, the following clauses can easily be audited in the regular course of reviewing other processes: 4.2.1), Quality System Documentation, 4.2.3), Document Control, 4.2.4), Record Control, 5.3), Quality Policy, 5.4.1), Quality Objectives, 6.2.2), Training, 6.3), Maintenance, 6.4), Work Environment, 7.1), Planning of Product Realization & Risk Management, 7.6), Calibration, 8.2.3), Monitoring & Measurement of Processes, 8.5.2), Corrective Action, and 8.5.3) Preventive Action. This strategy reduces the number of audits needed by more than half.

Internal Auditing: Upstream/Downstream Examples

Another way to embrace the process approach to auditing is to assign auditors to processes that are upstream or downstream in the product realization process from their own area. For example, Manufacturing can audit Customer Service to understand better how customer requirements are confirmed during the order confirmation process. This is an example of auditing upstream because Manufacturing receives the orders from Customer Service—often indirectly through an MRP system. Using this approach allows someone from Manufacturing to identify opportunities for miscommunication between the two departments. If Regulatory Affairs audits the engineering process, this is an example of auditing downstream. Regulatory Affairs is often defining the requirements for the Technical Files and Design History Files that Engineering creates. If someone from Regulatory Affairs audits these processes, the auditor will realize what aspects of technical documentation are poorly understood by Engineering, and quickly identify retraining opportunities.

One final aspect of the example audit schedule that I think can be improved is the practice of auditing the same process twice per year. This practice doesn’t seem to work very well for a few reasons. First, it requires that an auditor prepare for an audit twice per year and write two reports, instead of one. This doubles the number of time auditors spends in preparation and follow-up activities associated with an audit. Second, increasing the number of audits naturally shortens the duration of each audit. It is more difficult for auditors to cover all the applicable clauses in a shorter audit because it takes time to locate records and pursue follow-up trails. Longer audits, covering more clauses, make it easier for the auditor to switch to a different clause while they are waiting for information. Third, if an area is audited every six months, it is often difficult to implement corrective actions and produce evidence of effectiveness before the area is due for auditing again.

I can’t provide a generic audit schedule that will work for every company or even show how all the clauses will be addressed in one table. I can, however, provide an example of an improved schedule that illustrates the above concepts. This example (see below) uses four auditors instead of 12, and the number of days planned for each audit is two days instead of one. The preparation and reporting time is still one day per audit. Therefore the combined resources equal 24 days (~$10,000) allocated to auditing, and each person spends two and one-half percent of their work year auditing. My intention is not to create the perfect plan, but to give audit program managers some new ideas for more efficient utilization of resources. I hope this helps, and please share your own ideas as comments to this posting.

%name Improving Your ISO Internal Auditing Schedule

Posted in: ISO Auditing

Leave a Comment (5) →
Page 2 of 3 123