Blog

Posts Tagged auditing

How to Finish your Audit Schedule by December 31st

This blog provides viable options to consider related to successfully completing your audit schedule by year’s end.

Let’s say that there are 34 days until the end of 2012. You have four supplier audits and three internal audits to complete. Of course, all but two of these audits are overdue. What should you do?

Options that might be readily available to you include:

  1. Get some help
  2. Perform remote audits
  3. Reschedule some of the audits for next year

There are some great cartoons and jokes about doing more with less, but if you intend to complete seven audits before the end of the year, you might need some help. There really isn’t any time left to train someone, so that they are capable of conducting an effective audit by themselves. In fact, I expect training a new auditor to take at least six months before I believe they are ready to work solo. Even if you are less demanding than I am, you still would need time for classroom training and shadowing a couple of audits. Therefore, the best I believe you could hope for is one or two solo audits of the seven you need to complete.

Realistically, your only source of help would be auditors that are already trained and consultants. The last month of the year is historically very busy for everyone–especially quality assurance auditors. Therefore, consultants will not be cheap, and you should commit to any qualified consultants that are available without too much delay (then again, maybe they are available because they are not very good). If you have any in-house auditors that are already trained, do everything you can to get some of their time in the next few weeks.

Remote Audits

Option two is to perform remote audits. This is a viable option for you to justify for a supplier with an impressive quality track record, or suppliers in other countries. However, a remote audit is not the same as asking a supplier to complete a survey. ISO 19011:2011 provides some guidance specific to remote auditing in table B.1 of Annex B.

For a remote audit, you should still sample just as many records—if not more. You should conduct interviews by phone, Skype or some similar technology. You should analyze any available data to help identify which processes appear to be effective and which processes need to improve. If you are performing a remote audit for the first time, I recommend focusing on the same processes that you would normally audit in a conference room, rather than processes that you would typically audit where they occur—such as production controls. Regardless of which process you check, you should always request data.

Option three is to reschedule some audits for January 2013. I have suggested this so many times to clients, but very few follow this advice. If your company is late in conducting some audits, the important thing to do is to document this, reschedule the audits, and take corrective action(s) to prevent it from recurrence. If you wait until January, you will have additional time to train an auditor, as well. Finally, consultants historically have more time available in January than December.

In parallel with your efforts to catch-up on your schedule, I also recommend the following:

Create a quality objective that measures “on-time delivery” of audits and audit reports. This is an effective metric for managing an audit program.

Investigate the reasons for audits being overdue. If the occurrence was preventable, then I recommend initiating a CAPA. This will have two effects. First, your third-party auditors will see that you have identified the problem yourself and taken appropriate corrective action(s). If you also discuss this during a Management Review, this information can be used effectively to change the grading of an audit finding to a “minor,” or to potentially eliminate the finding altogether. Second, it will ensure that this doesn’t occur again.

Posted in: ISO Auditing

Leave a Comment (0) →

7 Steps to Auditing Design Controls Using the ISO 13485 Standard

This blog reviews seven steps to effectively auditing design controls utilizing the ISO 13485 standard.

Third- party auditors (i.e., – a Notified Body Auditor) don’t always practice what we preach. I know this may come as a huge shock to everyone, but sometimes we don’t use the process approach. Auditing design controls is a good example of my own failure to follow was it true and pure. Instead, I use NB-MED 2.5.1/rec 5 as a checklist, and I sample Technical Files to identify any weaknesses. The reason I do this is that I want to provide as much value to the auditing client as possible without falling behind in my audit schedule.

Often, I would sample a new Technical File for a new product family that had not been sampled by the Technical Reviewer yet. My reason for doing this is that I could often find elements that are missing from the Technical File before the Technical Reviewer saw the file. This gives the client an opportunity to fix the deficiency before submission and potentially shortens the approval process. Since NB-MED documents are guidance documents, I could not write the client up for a nonconformity, unless they were missing a required element of the M5 version of the MDD (93/42/EEC as modified by 2007/47/EC). This is skirting the edge of consulting for a third- party reviewer, but I found it was a 100% objective way to review Technical Files. I also found I could review an entire Technical File in about an hour.

So what’s wrong with this approach?

This approach only tells you if the elements of a Technical File are present, but it doesn’t really evaluate the design process. Therefore, I supplemented my element approach with a process audit of the design change process by picking a few recent design changes that I felt were high risk issues. During the process audit of the design change process, I sampled the review of  risk management documentation, any associated process validation documentation and the actual design change approval records. If I had time, I looked for the following types of changes: 1) vendor change, 2) specification change, and 3) process change. By doing this, I covered the following clauses: 7.4 (purchasing), 7.3.7 (design changes), 7.5.2 (process validation), 7.1 (risk management) and 4.2.4 (control of records).

So what is my bastardized process approach to auditing design controls missing? Clauses 7.3.1 through 7.3.6 of ISO 13485 are missing. These clauses are the core of the design and development process. To address this, I would like to suggest the following process approach:

Step 1: Identify the process owner and interview them. Do this in their office–not in the conference room. Get your answers for steps 2-7 directly from them. Ask lots of open-ended questions to prevent “yes/no” responses.

Step 2: Identify how design projects are initiated. Look for a record of a meeting where various design projects were vetted and approved for internal funding. These are inputs into the design process. There should be evidence of customer focus, and some examples of corrective actions taken based upon complaints or service trend analysis. Step 3: Identify where Design History Files (DHF) are stored physically or electronically, and determine how the DHF is updated as the design projects progress.

Step 4: This is typically the step of a process audit where there auditor needs to identify “what resources” are used in the process. However, only companies that have software systems for design controls have resources dedicated to Design and Development. I have indicated this in the following “Turtle Diagram.”

%name 7 Steps to Auditing Design Controls Using the ISO 13485 Standard

“With What Resources” is typically not applicable, because most companies do not have electronic design history files.

Step 5: Identify which people are assigned to the design team for a design project. Sometimes companies assign very large teams. In this case, the auditor should focus on the team members that must review and approve design inputs (see Clause 7.3.2) and design outputs (see Clause 7.3.3). All of these team members should have training records for Design Control procedures and Risk Management procedures.

Step 6: Identify procedures and forms that define the Design and Development process. Do not read and review these procedures. Auditors never have the time to do this. Instead, ask the process owner to identify specific procedures or clauses within procedures where clauses in the ISO Standard are addressed. If the process owner knows exactly where to find what you are looking for, they’re training was effective, or they may have written the procedure(s). If the process owner has trouble locating the clauses you are requesting, spend more time sampling training records.

Step 7: Ask the process owner to identify some metrics or quality objectives they are using to monitor and improve the design and development process. This is a struggle for many process owners–not just design. If there are any metrics that are not performing up to expectations, there should be evidence of actions being taken to address this. If there are no metrics being tracked by the process owner, you might review schedule compliance.

Many design projects are behind schedule and therefore this is an important metric for most companies. Now that you have completed your “Turtle Diagram”, if you have more time to audit the design process, you can interview team members to review their role in the design process. You could also sample specific Technical Files as I indicated above. If you are performing a thorough internal audit, I recommend doing both.

Posted in: Design Control

Leave a Comment (0) →

Auditing Medical Device Software Vendors

This blog presents some thoughts related to auditing medical device software companies.

Software medical devices are used to assist medical professionals. For example, radiologists use software with identifying areas of interest for medical imaging. Do you know how to audit a software company?

As a third-party auditor, I have had the pleasure of auditing software companies for CE Marking. When you audit a software company for the first time, this forces you to re-learn the entire ISO 13485 Standard. For example, if a company only produces software, there is very little to sample for incoming inspection and purchasing records. This is because the product is not physical—it’s software. Clauses of ISO 13485 related to sterility, implants and servicing are also not applicable to software products. If the software is web-based, the shipping and distribution clauses (i.e, – 7.5.5) might present a challenge to an auditor as well.

The aspects of the ISO 13485 Standard that I found to be the most important to auditing software products were design controls and customer communication. Many auditors are trained on auditing the design and development of software, but very few auditors have experience auditing technical support call centers. When auditing a call center, most of the calls represent potential complaints related to software “bugs,” system incompatibilities with the operating system or hardware and use errors resulting from the design of the user interface.

In most technical support call centers, the support person tries to find a work-around for problems that are identified. The problem with a “work-around” is that it is the opposite approach to the CAPA process. In order to meet ISO 13485 requirements, software companies must show evidence of monitoring and measuring these “bugs.” There must also be evidence of management identifying negative trends and implementing corrective actions when appropriate.

As an auditor, you should focus on how the company prioritizes “bugs” for corrective actions. Most software companies focus on the severity to software operations and the probability of occurrence. This is the wrong approach. Failure to operate is not the most severe result of medical device software failure. Medical device software can result in injury or death to patients. Therefore, it is critical to use a risk-based approach to prioritization of CAPAs. This risk-based approach should focus upon severity of effects upon patients—not users. This focus on safety and efficacy is an essential requirement of the Medical Device Directive (93/42/EEC as modified by 2007/47/EC) and is a requirement of ISO 14971:2007.

Posted in: ISO Auditing

Leave a Comment (0) →

7 Steps to Training Auditors on the Process Approach of the ISO Standard

The author uses the turtle diagram as the foundation to reviewing seven (7) steps to training auditors on the process approach for the ISO standard.

I have been reviewing trends for how people find my website, and a large number of you appear to be interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.

First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead you audit each process. Typical processes include:
  1. Design & Development
  2. Purchasing
  3. Incoming inspection
  4. Assembly
  5. Final Inspection
  6. Packaging
  7. Sterilization
  8. Customer Service
  9. Shipping
  10. Management Review
  11. CAPA
  12. Internal Auditing

Why the Process Approach is Recommended

First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments, the process approach will expose it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked.

Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e.,– the element approach). My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are actually incorporated into each process audit.

For example, each process audit requires a review of records as input and outputs. In addition, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment. The tool that BSI uses to teach the process approach is the “Turtle Diagram.” The diagram below illustrates where the name came from.

tutle diagram1 7 Steps to Training Auditors on the Process Approach of the ISO Standard

Process Auditing – “Turtle Diagram”

The Interview

The first skill to teach a new auditor is the interview. Each process audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next, most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique, because it is an “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore, I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.

After getting a general overview, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection, because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.

The next step of this approach is to “determine what resources are used by incoming inspection.” This includes gauges used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list, the auditor should select people to interview and follow-up with a request for training records.

The sixth step  is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures, this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

Process Owners Challenged

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company quality objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review monitoring and measurement of processes, and the trend analysis can be verified to be an input into the CAPA process.

In my brief description of the process approach, I used the incoming inspection process. I typically choose this process for training new auditors, because it is a process that is quite similar in almost every company, and is easy to understand. More importantly, however, the incoming inspection process does an effective job of covering more clauses of the Standard than most audits. Therefore, new auditors get an appreciation for how almost all the clauses can be addressed in one process audit. If you are interested in learning more about Turtle Diagrams and the process approach to auditing, please register for our webinar on the process approach to auditing.

Posted in: ISO Auditing

Leave a Comment (3) →

But What About FDA Regulations?

The author writes that when you are auditing, you should always read the FDA regulations again to ensure accuracy.  

I hear this question, or a question with similar wording, quite frequently when I am auditing. Typically, the question is in response to a better way to do something that seems simple and efficient. Most people seem to approach regulatory requirements with the approach of…let’s bury the regulator in paperwork. While it’s true that auditors expect a certain amount of paperwork with each regulatory requirement, they frequently accept a broader range of documentation than people realize (i.e., one page can be enough).

For example, a design control procedure could be a one page flowchart that references forms and work instructions, or twelve separate documents, with a minimum length of ten pages and a maximum of forty pages per document. As long as the procedure has sufficient detail for personnel performing these tasks, and all the required elements are included, ISO clauses 7.3.1-7.3.7, then an auditor should identify the process as conforming.

However, some people are FDA inspectors looking for NONCONFORMITY!

In the case of inspectors, it is critical to present your information in such a way that it is easy for the inspector to see how you meet the requirements of the regulations. One of the best ways to do that is to reference the requirements directly in your procedures.

For those that prefer finesse…try to organize information in accordance with the regulations. For example, if I am writing a procedure for an ISO registration audit, I write the procedure to specifically address the ISO sub-clauses. I might even use a document control number like: SOP-73 for my “Design and Development” procedure.

In my previous blog posting, http://bit.ly/AuditHours I suggested a slight change to the scheduling of internal audits. In order to ensure this meets FDA requirements, the key is to READ THE REGULATIONS AGAIN. With regard to internal auditing, the applicable FDA regulation is: 21 CFR 820.22:

“Each manufacturer shall establish procedures for quality audits and conduct such audits to assure that the quality system is in compliance with the established quality system requirements and to determine the effectiveness of the quality system. Quality audits shall be conducted by individuals who do not have direct responsibility for the matters being audited. Corrective action (s), including a reaudit of deficient matters, shall be taken when necessary. A report of the results of each quality audit, and reaudit(s) where taken, shall be made and such reports shall be reviewed by management having responsibility for the matters audited. The dates and results of quality audits and reaudits shall be documented.”

The above requirement is quite vague with regard to how many auditors and how many days must be spent auditing. These are the variables I suggested changing in my previous blog http://bit.ly/AuditHours. The FDA regulation 21 CFR 820.22 is  specific, however, with regard to documenting the “reaudit” of any deficiencies found during an audit. This prescriptive requirement can be met by reviewing previous audit findings of all audits with the audit program manager during the audit preparation process. The audit program manager can facilitate the assignment of which auditor will reaudit each finding. This may require a few more minutes of audit preparation, but this should not measurably impact the overall time allocated to an audit.

I do this out of habit when I am performing internal audits on behalf of clients, but if I am auditing the internal audit process of a client—now I’ll remember to point out this additional requirement that is specific to the FDA and not included in the ISO Standard. This is why we should always READ THE REGULATIONS AGAIN.

 

Posted in: FDA

Leave a Comment (0) →
Follow

Get every new post on this blog delivered to your Inbox.

Join other followers:

Simple Share Buttons
Simple Share Buttons