Supplier Qualification: How To Get The Best Results

1 Comment / Supplier Quality Management / By / ,

In this article, you will learn strategies for better supplier qualification to obtain the highest quality components and services.

Supplier Qualification in ISO 13485:2016

Section 7.4 of EN ISO 13485:2016 states that companies shall “evaluate and select suppliers… based on their ability to supply product in accordance with the organization’s requirements.” Supplier qualification is one of the most important purchasing controls. This requirement is quite vague, but the medical device industry has developed a surprisingly limited number of approaches to address the requirement of this clause.

The most common approach is to ask for some combination of the following:

  1.  ISO certification,
  2.  a copy of the Supplier’s Quality System Manual,
  3.  completion of a Supplier Questionnaire and
  4.  performing a Supplier Audit.

Unfortunately, all four selection criteria are flawed.

ISO Certification

I think the best way to explain why these criteria are flawed is to use an analogy. Let’s compare qualifying a new supplier with recruiting a new employee. ISO certification is sort of like a college degree. You can make some general assumptions about a potential job candidate based upon which school they got their engineering degree from, but the degree is still just a piece of paper on the wall. As the old joke goes:

           ” What do you call the person that graduated last in their class at medical school?

Doctor

Some registrars have a better reputation than others. Still, the name of the registrar is only as good as its worst client—who had four major nonconformities during their last audit and is about to lose that certificate. To improve this approach to supplier qualification, a potential customer could ask for a copy of the most recent audit report. This information is dependent upon the quality of the audit, but this would be a significant improvement over requesting a copy of the certificate.

caution sign picture warning 6699085 960 720 Supplier Qualification: How To Get The Best Results  CAUTION: Audits are still just samples—tiny samples. 

Again, like degrees, certification must be relevant. ISO 9001:2015 may be a ‘nice-to-have’ quality for potential suppliers. However, it doesn’t hit the mark if you need them to have ISO 13485:2016 certification. Perhaps you need a European Normative version, or A11:2021 as well. For example, sometimes any law degree might be appropriate. Sometimes you specifically need a degree in healthcare law. 

This makes it important to establish the criteria for your supplier evaluation early on in the process. Not just because it is required for standard compliance. It is difficult to evaluate a supplier with no guidance on how or what to evaluate them against. 

Supplier Quality Manual

The second selection criteria mentioned is The Quality Manual. The Quality Manual is analogous to a resume. The purpose of a resume is two-fold: 1) to provide an interviewer with information, so they can ask the interviewee questions without looking like an idiot, and 2) to provide objective evidence that a company did not illegally discriminate against a candidate that the hiring manager did not like.

I suppose you could argue that the purpose is to help candidates get a job, but in my own experience, less than 10% of resumes submitted result in a job interview—let alone a job offer. The purpose of a Quality Manual is NOT to help a company get new customers. If I am wrong about this, I need to do a much better job of marketing my Quality Manuals in the future.

Some suppliers have the nerve to say that their Quality Manual is proprietary. Humbug! Proprietary information should not be in the Quality Manual. You can copy a manual from another company and edit a few of the details. I will gladly write you a Quality Manual in less than a week that will pass any auditor’s review. You can even buy a Quality Manual online (In fact, Medical Device Academy sells one… Online! POL-001 Quality Manual). This almighty document just explains the intent of the Quality System—which is to conform to the requirements of the ISO Standard. Several auditors will tell you that this can be done in just four pages.

When you request a Quality Manual from a supplier, your primary intent for supplier qualification should be to use this document for planning a supplier audit. Any other purpose is just a waste of your time—unless you need to write a Quality Manual of your own.

Supplier Qualification Questionnaire

The third selection criteria I mentioned was: a supplier questionnaire or supplier survey. Questionnaires are analogous to employment applications. Coincidently, supplier questionnaires are often required by companies when a Quality Manual or ISO Certificate is not available. Do you find the similarities eerie?

Questionnaires are typically 15-20 page documents that someone has plagiarized from a previous employer. I have seen various versions of this questionnaire, but several of them appear suspiciously similar. Hmmm?

I am not sure what the original intent of this type of document was, but I think it was intended to capture detailed information about potential suppliers for a company in the Fortune 500®.

For most companies, 80% of the information on the questionnaire is meaningless. Customer requirements for a supplier are typically few in number and specific to the product or service being purchased. Therefore, please use your MRP system as a template and ensure that the questionnaire answers all the information you need to add the supplier to your system as an approved supplier. You should also have a product or service specification that gives you some more questions to ask.

Ideally, your questionnaire will be organized in the same order that you enter the information into the MRP system. Then this questionnaire will make the data entry easier for the purchasing agent, adding the supplier to the database. Questionnaires and surveys are great, but brevity is next to Godliness.

Supplier qualification questionnaire Supplier Qualification: How To Get The Best Results

Supplier Qualification Audits

Finally, we come to the auditor’s favorite—supplier audits. Audits are similar to job interviews. Ideally, you want a cross-functional audit team, and you might need to visit more than once. Unfortunately, most companies cannot afford to audit every supplier. Some companies supplement with remote audits. I guess I think of a desktop audit as a “phone interview.” I use phone interviews to prescreen candidates before I pay more money and waste other people’s time with on-site interviews. Desktop audits of suppliers should not be used as a replacement for an on-site audit, so your supplier quality engineers do not have to spend so many nights at the Hampton Inn.

If audits are your best selection criteria, how can you make the most of your auditing resources? Also, how can you audit for supplier qualification if you only have enough auditors to audit 5% of the approved supplier list? I have the following suggestion: “Start at the end.” You might consider reviewing our article on hiring an auditor.

ISO 13485:2016 Clauses 8.5.2 / 8.5.3  CAPA

What I mean by this cryptic, four-word phrase is that auditors should start at the end of the ISO Standard with sections 8.5.2 & 8.5.3 (Corrective and Preventive Action (CAPA) Process). This is the heart of a Quality System. If you disagree, remember that FDA inspectors are required to look at the CAPA system during every Level 1 inspection. Registrars also look at the CAPA process during every assessment—not just the certification audits. The purpose of the CAPA process is to fix problems, so they don’t come back—ever.

If you think that a new supplier is never going to make a mistake, you might as well quit looking. You want suppliers with strong CAPA systems. If a supplier has a strong CAPA system, problems will be fixed quickly and permanently. To sample the CAPA process, an auditor only needs the following: 1) the CAPA procedure(s), 2) the CAPA log(s), and 3) a handful of completed CAPA records—selected not so randomly from the log(s). This can all be done remotely in a desktop audit. If suppliers are resistant to giving you the log or actual records, ask them to redact any sensitive information. If you have executed a nondisclosure agreement, the supplier should agree with this approach.

ISO 13485:2016 Clause 8.4 Analysis of Data

Working from the back of the Standard, the next process to sample is clause 8.4 (Analysis of Data). There are four requirements of this clause. If the company has a requirement for customer satisfaction to be measured (ISO 9001:2008 section 8.4a), this is a great place to focus. There are also requirements to look at the trend of product conformity (8.4b), process metrics (8.4c), and trends in supplier data—such as on-time delivery and raw material nonconformities (8.4d). The quality of the analysis will tell an auditor as much about the company as the data itself. This process audit can also be performed remotely as a desktop audit.

A lot has changed since this article was first written. For example, if your potential supplier isn’t using ISO 9001:2015 you may want to verify that other areas of their quality management system aren’t outdated as well. 

ISO 13485:2016 Clause 8.3 Control of Nonconforming Materials

Clause 8.3, Control of Nonconforming Materials, is the third area to look at. To sample this area, you will need the “Holy Trinity” again: 1) procedure, 2) log, and 3) records. In this desktop audit, you want to look very closely at any nonconforming materials that are reworked or accepted “as is” (i.e., UAI). Either of these two dispositions should be ULTRA-RARE. Everything else should be processed efficiently as scrap or returned to the Vendor (i.e., – RTV).

If a potential supplier passes all three “tests” described above, you are ready to address clause 8.2.4—Monitoring & Measurement of Product. In this section, there is a requirement to maintain records of product releases and to verify that product requirements are met. for supplier qualification, if you think you can effectively audit this by paperwork alone, the supplier is a good candidate for “desktop only.” However, if the lot release paperwork, batch record, or Device History Record (DHR) is a 50-page tome—then you better make your flight plans.

The good news is that very few suppliers will pass the first three tests and implode during the on-site audit. Also, with three process audits complete, you should be able to reduce the duration of your on-site audit. Finally, for low-risk suppliers, you have a strong basis for provisional approval of suppliers to proceed with prototype runs before you schedule an on-site audit. If you need a procedure for supplier qualification, please check our Supplier Quality Management Procedure (SYS-011).

Medical Device academy Youtube Button 1024x451 Supplier Qualification: How To Get The Best Results

For more information on supplier controls, quality systems, auditing, and regulatory submissions visit our YouTube Channel!           

 

Supplier Qualification: How To Get The Best Results Read More »

Why remote audit duration should never exceed 90 minutes

1 Comment / Auditing, ISO Auditing, Remote Auditing / By

This article explains why remote audit duration should not exceed 90 minutes and the unique opportunities created by a series of short remote audits.

download 3 Why remote audit duration should never exceed 90 minutes

Parkinson’s Law and the subject of audit duration

On November 19, 1995, Cyril Northcote Parkinson published an essay in the Economist. The title of the article was “Parkinson’s Law.” In the first sentence of the essay, Parkinson says, “It is a commonplace observation that work expands to fill the time available for its completion.” This essay refers to the observation that work is elastic concerning the demands on time when completing paperwork. When I first trained as an auditor, trainers emphasized that the most significant challenge faced by auditors is to complete an audit within the time available. An auditor’s task is to achieve the audit objectives within the time specified by the audit program manager. Time is precious, and you cannot easily extend the audit duration after scheduling the audit.

How much time is needed for a full quality system audit?

This question is a silly question to ask a consultant that works on an hourly basis. A consultant working on an hourly basis will make more money if they work more hours. Therefore, there is little incentive to underestimate the time required to complete the objectives of an audit. However, after completing hundreds of audits, I can honestly state that eight hours is not enough time to perform a full quality system audit of a medical device company’s quality system. However, I completed a full quality system audit of a small company in less than two days. I also had difficulty completing an audit of a larger company in four days. An FDA inspector typically requires four days to complete a routine inspection, even at foreign manufacturers where English is a second language, and they only need to return on the fifth day to prepare their FDA 483 observations to give to the company. Therefore, three days is typically the absolute minimum time required to complete a full quality system audit.

Does Parkinson’s Law apply to audit duration?

Parkinson’s Law certainly applies to the audit duration. If the lead auditor assigns a team member to review the CAPA process, the task is unlikely to be completed in 30 minutes, and most auditors would struggle to appear busy for more than three hours. You need enough notes to provide objective evidence of conformity for your audit report, but if you finish too quickly, then others may perceive that you were not thorough. Therefore, most auditors will begin any process audit by asking for a copy of the procedure and a log of the records available. The auditor will quickly review the procedure’s revision history to determine when the last revision was made and if there have been any significant revisions since the last audit. Next, the auditor will review the log to estimate how many records should be sampled. The auditor will then estimate how much time is needed to review the sampled records. Finally, a quick mental calculation is made to determine how much time remains for procedure review before the auditor must move on to interview the next subject matter expert.

Why are auditors always behind schedule?

An auditor begins with small, close-ended questions that are designed to put the auditee at ease. The auditor may even comment on unrelated subjects to build rapport first. Records may not be readily available, but auditors almost always have to wait for record retrieval. The request is recorded, copies are made, and the subject matter expert may need a little time to review before handing the auditor the requested record. Auditors will ask clarifying questions, and auditees will need a few moments to check their facts. Any one of these delays is insignificant by itself, but collectively there may be two-and-half minutes of delay cumulatively for each record requested if you sample five records, which represents a combined delay 12.5 minutes. If you average only seven minutes to review each record, then a sampling of five records will require 47.5 minutes. This will leave you only 12.5 minutes for introductions, review of the procedure, and conclusions. If you want to interview any of the people that investigated root-cause, then you will need more than an hour to complete your audit, and you will not finish in the one hour scheduled.

Why is it so hard to complete a full quality system audit in three days?

Most of your process audits require a few more minutes than you expected, but you will also need time to walk to the next subject matter expert, or you will be waiting for the next subject matter expert to enter the conference room. If the quality system consists of only the minimum twenty-eight required procedures, your full quality system audit will require more than 28 hours to complete. If there are additional regulatory requirements for CE Marking or ISO 13485 certification, you will need even more time to audit every process. You should also expect certain processes to require more time to properly sample records, such as technical documentation and design controls. Even the most experienced auditors struggle to review a technical file and/or design history file in less than two hours.

What happens to an auditor after auditing all day?

As a Notified Body auditor, I used to leave my home in Vermont on Sunday afternoon and drive two hours to the nearest major airport. Then I would be gone all week conducting audits. On Friday, I would drive home and arrive in the middle of the night. Each day audits would begin early in the morning, and I would complete the day after 8.5 to 9 hours of work. Jet lag, sleep deprivation, too little exercise, and constantly eating at restaurants took its toll. I would consult my Google calendar to learn what city I was in each morning, and to remember what company I was on my schedule for the day. I would purposely try to do as much walking around during the day just to keep my blood flowing and to help stay awake. I would read documents while pacing back-and-forth in conference rooms, and I would always make sure that we had to audit the most remote area of a facility after lunch to make sure that I didn’t fall asleep. I will tell stories and jokes to entertain my hosts, but it was necessary to break up the monotony of auditing quality systems seven days a week. I would make sure I drank at least six liters of water each day for health, but this also gave me an excuse to go to take frequent bathroom breaks. Somehow I managed to survive that lifestyle for more than three years. Each day my feet, legs, back, and neck were in severe pain. I had constant headaches, and I know the quality of my work gradually declined throughout each day. The most valuable lesson I learned was, you need to move frequently, or you will die.

unnamed Why remote audit duration should never exceed 90 minutes

What happens when you sit in front of a computer for eight hours?

I can sit in front of a computer longer than almost anyone I know. When I focus on work, four hours can elapse without me getting up from a chair even once. I might pick up my empty coffee mug four or five times to take a sip before I am conscious of the need to get another cup. On days where my schedule consists primarily of Zoom meetings, I may sit through as many as six consecutive meetings before I take the time to get up and go to the bathroom and get a drink of water. Clients may perceive that I have tremendous endurance, but there are negative consequences to this work pattern. My wrist becomes sore, and I need to switch my mouse pad and the style of the mouse I am using every day. I change computers, switch microphones, and take a short walk. My neck, back, and legs will hurt worse than any of the audits during my years as a Notified Body auditor. Sitting at a computer all day has resulted in mild symptoms of restless legs syndrome. Sitting at a computer continuously for the audit duration is physically exhausting and tedious. If you must complete a remote audit on a continuous eight-hour day, you can, but it is not healthy or productive. The negative health consequences and negative impact on productivity are equally applicable to auditees.

What can you do to reduce audit fatigue during a remote audit?

The most straightforward strategy for reducing fatigue is to take breaks. Instead of auditing for eight hours continuously, try auditing in two or three 90-minutes segments each day. If you are auditing someone in a different time zone, you may only be able to accommodate an audit duration of one 90-minute session per day without working through the night. Taking breaks will allow you to leave your computer, eat food, and even go to the bathroom. You can recharge your headset during a break too. You should consider taking a walk outside. It is incredible how much better you feel when you get some exercise, stretch, and experience a little natural light instead of the unnatural glow of your computer’s monitor. The person you are auditing will appreciate the breaks, but they will also enjoy the improvement in your overall demeanor. A simple smile after a 30-minute break has a tremendous positive impact.

How can we utilize breaks more effectively during remote audits?

Auditors need documents and records to review as objective evidence. The most obvious way to make use of breaks is for the auditor to give the auditee a list of documents and records to gather during the break. This will give the auditee an excuse to go and get the documents and records if they are stored in another location. The auditee might also scan records during a break. A break also gives subject matter experts time to re-familiarize themselves with the documents and records before resuming the audit. Auditees and auditors will need to recharge batteries, but the auditor might take time to convert their notes into a summary for the final audit report. The auditor might also review the audit criteria one more time before writing a nonconformity. The auditee might take advantage of the break to initiate a new CAPA and write a draft of the corrective action plan. Then when the audit resumes, the auditee can review the draft plan with the auditor to ensure that the plan is appropriate and nothing was accidentally omitted from the CAPA plan.

unnamed 1 Why remote audit duration should never exceed 90 minutes

Why are 90 minutes a magical audit duration?

Auditing one process in a single 45-60 minute session is ok, but if you audit two processes in a single 90-minute session, you can reduce the time spend starting and stopping the audit session by half. Adding a third process to a single session will have a smaller impact, and the meeting will need to be so long that most participants will begin to lose concentration, and fatigue becomes a significant factor. Ninety minutes is not quite long enough to audit two processes effectively. Still, an auditor can request procedures in advance of the session or spend time after the session reviewing procedures. Therefore, by paying an additional 30 minutes reviewing two procedures “off-line,” the auditor can dedicate 100% of the “on-line” time to reviewing records and interviewing subject matter experts. The result is a fast-paced, 90-minute session where each subject matter expert typically is only needed for 45 minutes. Alternatively, if you are auditing more complex records like a design history file, you can spend all 90 minutes discussing that area.

Why remote audit duration should never exceed 90 minutes Read More »

How to make a supplier questionnaire for remote auditing

Leave a Comment / ISO Auditing, Supplier Quality Management / By

You already have a supplier questionnaire, but do you know how to make a supplier questionnaire to assess a supplier’s ability to support a remote audit?

FRM 004 Supplier Questionnaire How to make a supplier questionnaire for remote auditing

The four most significant mistakes people make when designing a supplier questionnaire

In Medical Device Academy’s supplier qualification webinar, you learn how to improve your supplier qualification process by replacing the traditional methods of supplier qualification with more effective approaches to supplier evaluation. The following are four examples of how to improve your supplier questionnaire.

Supplier questionnaires should be specific to the product or service provided

The first mistake people make is to use a generic questionnaire. It would be best if you asked your supplier questions that are important to the work that the supplier will be performing. Therefore, each category of product or service should have its own set of questions. For example, important questions related to ethylene oxide contract sterilization services are the maximum size limitations for pallets in the sterilization chamber and whether the facility can conduct sterility testing on-site. However, an injection molding supplier might delay the return of your supplier questionnaire if these questions were on the survey that you send to them because they don’t understand the questions.

Supplier surveys should be more than checkboxes

The second mistake people make is to ask questions that can be answered with a “yes” or “no” response or a checkbox. These are closed-ended questions. It would be best if you always were asking open-ended questions because the response will give you more information about the supplier. In addition, most people resist responding with a “no” response even if the real answer is “no.” For example, “What is your FDA registration number?” is more useful than “Is your company FDA registered?” Another example is, “How many production lines use SPC charts?” instead of “Do you use SPC charts?” In fact, in the open-ended version of this question, you will learn if the use of SPC charts is widespread, and you learn how many production lines the supplier has.

Remember to ask suppliers to update survey surveys every year

The third mistake people make is to request that a supplier questionnaire be completed only during the initial supplier qualification process. Every year companies grow, shrink, or change. If you ask suppliers to update their questionnaire, you can use that information to determine the health of your supplier’s business. You might also discover that one supplier just added a new production capability that will allow you to consolidate more of your outsourced work with that supplier and eliminate another problem supplier. Every company has a turnover in personnel as well. It is a great idea to ask suppliers to provide contact information for multiple people in the organization, such as quality contact, billing contact, and a production planner. Eventually, you will probably need to speak with each of these people, and if one of the contacts is no longer at your supplier, you will still have two other contacts. Updating this information also gives you a hint of whether turnover is widespread or limited to a specific individual.

Supplier questionnaires should be in spreadsheet format

The fourth mistake people make is to send a Word Document for suppliers to complete (PDF format is even worse). Word and PDF formats are time-consuming to complete, and they are harder for you to analyze than a spreadsheet. Most people provide a Word document or a PDF because they are focusing on the requirement for control of records. However, if you have an electronic quality system, the supplier survey information will be part of your electronic system as soon as you enter the data into your software. Alternatively, if you have a paper-based quality system, then you can print the spreadsheet out, sign it, and date it. The huge advantage of using Excel spreadsheets is that you can copy the new data into a column next to the previous year’s responses. Then you can quickly see what changes your supplier made in the past year.

What should you add to your supplier questionnaire?

Most private companies will not share what their revenues are for the business, but as a customer, you should be more concerned with how many human resources your supplier has. Therefore, you should consider asking, “How many employees, or full-time equivalents (FTEs), work for your company?” You might also want to know if your supplier is relying on a temporary workforce. For example, “What percentage of the FTEs are temporary workers?” Many questionnaires will ask for the square footage of the facility, but this doesn’t provide you with any details about the facility layout. Alternatively, you could ask for a copy of the pest-control map for the facility. This would give you a detailed layout of the facility, and it also confirms that your supplier has a pest control plan for the facility. Another related question to ask is, “Please describe any expansion/construction projects that have been implemented in the past year or projects that are in progress (e.g., the addition of a mezzanine).” If the company added 30,000 square feet to their production area, but there was no change to the pest control plan, you might have some clarification questions for your supplier. In general, a good strategy for developing your questionnaire is to think of at least one open-ended question related to each clause of the ISO 13485:2016 standard without referencing the standard. The following are some examples that might help you:

  1. When was the last software re-validation for quality system software?
  2. How many active external standards is your company currently maintaining?
  3. Please provide a list of procedures and identify the person who would be interviewed during an audit for each procedure (i.e., process owner or subject matter expert).
  4. In the absence of the management representative, who is designated as the liaison for an FDA inspector?
  5. What are the upper control limits for particulate counts, air viable counts, and surface viable counts in your controlled environment(s)?
  6. On what dates was the environmental monitoring of controlled environments conducted in the last year?
  7. Please identify how many quality inspectors are responsible for the incoming inspection?
  8. Please list the calibration ID and equipment name for any inspection equipment that requires specialized training (e.g., CMM)?
  9. How many suppliers are on your approved supplier list (ASL)? And how many suppliers did you audit in the past year?
  10. How many nonconforming material reports (NCMRs) were opened in the past year? And how many NCMRs currently remain open?
  11. How many partial or complete lots were returned to your company by customers in the past year?
  12. Please list any corrections and removals (i.e., recalls) that your company has been involved in during the past year and the current status?

How many questions should your supplier questionnaire include?

There are 28 required procedures in ISO 13485:2016, and there are even more subclauses within the standard. It is an excellent idea to create a list of questions you might ask for each subclause, but a supplier questionnaire should not include all of those questions. Just as audits are just a sampling, your supplier survey questions should be sampling as well. You should review last year’s questions and eliminate questions that you think are not especially useful for that supplier. Some questions should be asked each year to assess if the quality system has changed significantly, and you should consider adding a few new questions each year. The best questions will require the person to perform some research to answer the questions. But it is unreasonable to expect a supplier to spend more than two hours completing a supplier questionnaire if you plan to purchase less than $20,000 in product or services.

Supplier questionnaires specific to remote auditing

In many ways, a well-designed supplier questionnaire is similar to a remote audit, because you are asking the supplier to answer multiple open-ended questions about their quality system to verify that the quality system is fully implemented and remains effective. However, due to the Covid-19 pandemic, many employees are now required to work from home, and it is not possible to physically visit certain facilities. Therefore, you should be adding three elements to your supplier questionnaire to assess your supplier’s ability to support a remote audit and to determine their ability to maintain the effectiveness of the quality system during a viral outbreak. The three elements are 1) policies for personal protective equipment for employees and visitors, 2) business continuity plans to maintain internal operations and to ensure redundancy of crucial suppliers, and 3) availability of digital documents and records or paper documents and records via video conference software. These three areas were also the subject of a previous blog on changes triggered by Covid-19. It would help if you also asked about the availability of hardware and software communication tools for conducting a remote audit. You might ask your supplier, “Which areas of your facility can we observe during a remote audit using live video conferencing (e.g., Zoom mobile application)?” and “What experience does your company have in the use of Zoom as a video conferencing tool?”

Gimbal How to make a supplier questionnaire for remote auditing

Access to documents and records during remote audits

During a remote audit, you will need to access documents and records virtually. If your supplier can participate via a video conferencing tool with a high definition web camera or smartphone, then you should be able to see any documents and records that you could normally see during an on-site audit. However, your supplier will need to hold the document or records steady, possibly by using a music stand and a camera tripod so that you can take notes regarding the contents of the document or record. You will also need a way to record your notes. You might try using a Pixelbook or similar computer to write your audit notes. At the same time, you watch the video conference using a second computer–possibly on a conference room projector screen or large flat screen monitor. You could also use a tablet, such as remarkable. Of course, you can always use a pad of paper and a pen and then transcribe your notes later. All of these methods will be faster and more convenient than digitally scanning each document and uploading the documents to a shared folder or sending the scanned document by email.

It would help if you also were asking your supplier which records are already available digitally. You can expect all of the quality system procedures to be available in digital formats, but many records may already be available electronically as well. For example, purchase orders, quality system certificates, drawings, and blank forms should be available in digital format. In a supplier audit, you typically will focus on a subset of the quality system records that are related to production process controls, purchasing, incoming inspection, shipping, and control of the nonconforming product. Asking your supplier which of these records are available in digital format will help you determine which records you need to request from the supplier in advance and which records can be requested on-demand.

How to obtain our supplier questionnaire template (FRM-004)

If you are interested in purchasing our supplier questionnaire template, FRM-004, it is included with the purchase of our supplier qualification webinar. If you think of any new questions to add to this template, please email me at rob@13485cert.com. Just put “FRM-004 Suggestion” in the subject line.

How to make a supplier questionnaire for remote auditing Read More »

How to avoid the most common supplier evaluation mistakes

1 Comment / Supplier Quality Management / By /

The focus of this article is on the process of supplier evaluation and re-evaluation for medical device companies and how to document your evaluations.
No Grandfathering Image How to avoid the most common supplier evaluation mistakes

You have several suppliers today, but did you have a rigorous supplier evaluation process when you first hired those suppliers? If your business is going to be successful, you need to treat your supplier evaluation process as a critical strategic process. Supplier qualification and is more important than the hiring of any senior manager. ISO 13485:2016 requires you to have a procedure for supplier evaluation and re-evaluation, but the type and extent of your supplier controls are not specified.

Which of your suppliers are critical or crucial?

Crucial suppliers were defined in a draft policy published by the European Commission as part of the introduction of the requirement for unannounced audits. Essential suppliers make a component or subassembly that is high-risk, or your firm cannot easily purchase the component or subassembly from another supplier. Critical suppliers for medical device manufacturers fall into one of three categories: 1) a contract manufacturer, 2) a contract sterilizer, or 3) a contract packager or labeler. These three types of suppliers may be selected for unannounced audits by a Notified Body. The FDA also requires these three categories of suppliers to register their facility.

Should you establish other supplier evaluation categories?

The short answer is no. The purpose of categories is to ensure that a large number of suppliers are consistently managed. Instead, try reducing the number of suppliers you are managing. Give your best suppliers more work, and fire the worst suppliers. If a component is “single-source,” encourage another supplier to quote that business before you look for a new supplier. It would be best if you took the time to evaluate each supplier thoroughly. If you don’t have the supply chain resources to do this, then you have three choices: 1) hire another person to help manage your supply chain, 2) fire suppliers that are not meeting your requirements, or 3) replace the weakest member of your supply chain team.

How do you re-evaluate existing suppliers now?

There are a lot of possible answers to this question, but unfortunately, the most common answer is, “because that’s who we’ve always used.” This practice, referred to as “grandfathering,” is a horrible approach to supplier re-evaluation. Suppliers that miss your requested delivery dates, and suppliers that ship nonconforming product should be required to implement supplier corrective actions immediately. You need to follow-up on these corrective actions and verify that the corrective actions were effective. If the corrective actions are not effective, or if new supplier issues occur, then you should find an alternate supplier as soon as possible.

Another stupid reason for selecting a supplier is “because they were the lowest bidder.” There’s an old government contracting joke about this strategy. It sounds something like this, “a million mission-critical parts, designed by engineers that have no clue what the real world is like, built by the lowest bidder, and inspected by a bureaucrat that can be bribed with a bottle of wine and some prime rib.” I tend to discount the quality of the lowest bidder every time. I always wonder what they forgot to consider when they bid on the job. If the lowest bidding supplier can explain why they have an inherent advantage over their competition, then maybe you should consider hiring them. If there is no rational reason why a supplier’s pricing is below their competition, this usually means that the supplier is desperate, or they plan to increase their pricing after you are a customer.

What should be your supplier evaluation and re-evaluation criteria?

All medical device suppliers should have a quality system, but ISO certification is not required. Therefore, if a supplier has ISO 13485 certification, you might abbreviate your initial supplier qualification process. However, ISO 13485 certification should have minimal impact upon your on-going supplier evaluation process. You need to know how well your supplier’s quality system is being maintained. If your supplier is sharing copies of their annual surveillance audits and FDA inspection reports with you, this will give you a better indication of the quality system effectiveness.

Consider performing supplier audits for supplier evaluation

Although it is not required, the best way to evaluate the effectiveness of a supplier’s quality system is to perform a supplier audit. Specifically, you should focus on the processes that are directly related to your product or component. Production process controls and final inspection are the most critical areas to audit. Other areas that are important to consider for supplier audits are 1) incoming inspection, 2) purchasing controls, 3) shipping, and 4) control of nonconforming materials. Conducting a supplier audit using the process approach is the most effective method. The process approach method of auditing will ensure that document control, record control, calibration, process validation, and training are sampled as support processes. The supplier audits may also be conducted as on-site audits or remote audits.

Certificate of Conformity (CoC) vs. Certificate of Analysis (CoA)

Another supplier evaluation criteria should be product conformity. You should be reviewing more than whether your supplier shipped the correct product and the correct quantity. Did your supplier provide a Certificate of Analysis (CoA) that summarized the inspection methods, acceptance criteria, and the inspection results? Or do you verify that a Certificate of Conformity (CoC) was included and accept the lot you received? If your company is only receiving a CoC from a supplier, you should be sampling the product at incoming inspection and verifying that the product is conforming with your requirements. Even if the supplier is providing a CoA, you should still perform periodic sampling and inspection of the product to make sure the CoA provided matches the actual product you are receiving.

Considering Improving your supplier questionnaires

If your company is requesting that suppliers complete supplier questionnaires, make sure that you are asking the most relevant questions. You need to know if your supplier can support remote audits. You need to know if there have been any significant changes to the quality system. You need to know if the company has had any significant non-conformities resulting from certification body audits. You need to know if there have been FDA inspections and what the results of the inspection were. You should also be obtaining monitoring and measurement data related to process conformity and product conformity. Asking your supplier to identify any shutdown periods or planned renovations is a required input for critical and crucial suppliers for CE Marked medical devices subject to unannounced audits. It would help if you also were asking your supplier to update the names, titles, and contact information for key management personnel. Would you like a copy of our supplier questionnaire?

No Grandfathering Image 1 e1591549101295 How to avoid the most common supplier evaluation mistakes

What should you be doing to address the Covid-19 pandemic?

As a consequence of the Covid-19 pandemic, many suppliers have had significant disruptions to their supply chains, workforce availability, and transportation vendors. Since many medical device products are urgently needed during this pandemic, it is important to ask suppliers to provide a summary of their current situation and any analysis they have done to assess potential risks that could disrupt your supply chain. Does your supplier have adequate personal protective equipment (PPE)? What type of precautions is being taken to ensure that employees don’t exhibit symptoms of Covid-19 illness? Does your supplier have a policy for self-quarantining if an employee is exposed to someone that has the virus? Does your supplier have a disaster recovery plan?

Consider using size for supplier evaluation

Bigger is not always better. If you are a small customer of a large supplier, your needs will seldom be important to your supplier. Alternatively, if your company is much larger than your supplier, your supplier may not have the resources to grow with you and keep up with your current demand. When you are initially qualifying suppliers, try to select suppliers that are approximately the same size as your company or slightly larger. You should also consider identifying more suitably sized suppliers if you have a significant size mismatch or one develops over time.

What if you don’t have the resources to evaluate your suppliers?

Supplier evaluation and re-evaluation is a strategic function that impacts your profits, your ability to deliver product on-time to your customers, and nonconforming product can tarnish your company’s reputation. Therefore, your company needs to invest resources to analyze your supply chain. It would help if you had suppliers that have excellent quality and suppliers that will encourage your company to improve. Are there best practices you can learn from your suppliers? Is your supplier able to help you manage your inventory? Can your suppliers help you solve production problems? Supplier evaluation should only be secondary in importance to your design process and post-market surveillance. As they say, “garbage in equals garbage out.”

Do you need additional training on supplier evaluation?

On June 25, 2020, at 11 am EDT, and we are hosting a live webinar on how to qualify your suppliers. In this webinar, you will learn how to qualify new suppliers even if they don’t have ISO certification and best practices in supplier evaluation. We will be sharing a new supplier questionnaire that includes questions to help you assess whether a supplier is capable of supporting remote audits. We will help you develop a strategy for the allocation of supply chain personnel, and show you how to convince top management to prioritize supplier audits.

How to avoid the most common supplier evaluation mistakes Read More »

Remote audit resources – software and hardware tools

2 Comments / Remote Auditing / By

If you are planning a remote audit, you will need more remote audit resources than a webcam and web conferencing software. Matthew Walker is a significant contributor as co-author of this article.

20200525 095104 scaled Remote audit resources software and hardware tools

Clause 5 of ISO 19011:2018 is titled “Managing an audit program,” and subclause 5.4.4 is specific to determining audit program resources. For conducting audits remotely, you will need remote audit resources. Almost every laptop has a built-in webcam and microphone, and that is the minimum functionality you will need to conduct a remote audit. However, adding other software and hardware technology can improve the efficiency and effectiveness of your audit team.

What remote audit resources do you need?

Remote audits are not the same as a desktop audit, because a remote audit requires remote access to more than emails containing procedures and records. Auditors need access to people and access to physical areas of your facility. This creates one of the most significant challenges for this type of audit method. Call me a Negative Nancy, but I suspect that most audit plans do not specifically include logistical preparations to support this audit method. On the surface, it seems like a simple concept. Internet access and a scanner should cover most of the needs for the auditee to survive this digital encounter. In practice, conducting a remote audit that genuinely adds value and does more than checkboxes, requires serious planning.

Let’s start with the obvious; a remote audit needs a way for the auditor and the auditee to communicate with each other. Ideally, you need more than your phone. We recommend Zoom for video conferencing, but we list several other video conferencing software applications below. Here are the features of Zoom that we typically use during a remote audit:

  • Video Chat – Using Zoom, two or more parties can communicate using video input from webcams. This is nice because it allows for a more visual conversation, and you can see more of the facial expressions and body language of the person you are speaking with than you can with a traditional phone call. It also allows for sign language to be used if necessary.
  • Screen Sharing – Screen sharing is an essential tool you will use during a remote audit because it allows you to share documents and records on your screen even if you are not the host. The more records you have electronically, the more valuable screen sharing will be during the audit. An auditor can say, “Can you show me that quality system certificate again?” or “Can you show me where Isomedix is on your approved supplier’s list?” Being able to facilitate those verification activities saves the auditee the hassle of emailing documents or uploading content to a shared folder. This ability to share your screen is also essential for an auditee to demonstrate training effectiveness and competency.
  • Recording – Meetings can be recorded in their entirety or sections. This allows the auditee to record the opening or closing meetings of the audit to share with others that were unable to attend. If there are questions regarding non-conformities or opportunities for improvement, a recording of the conversation ensures that the auditor has an accurate record of complex objective evidence that would slow down the audit and gives managers a perfect record to demonstrate the issue when corrective actions are initiated.
  • Chat Record – Zoom, and most other video conferencing software, provides a chat box that can be used to take notes. If someone runs to the bathroom, and you don’t want to forget your question, you can enter it in the chatbox. Chat boxes are especially helpful when there is a language barrier, or someone’s accent is hard to understand. Text typed in the chatbox also serves as a place to record information that may be difficult to remember if you cannot access your audit report. If a production area has too much background noise, the chat feature might be the best way to communicate important details, such as: “That information is found in section 7.5.6 of the Quality Manual; POL-001 rev A.” The chatbox can also be used to communicate a list of documents, or records in a specific date range, that you want an auditee to make available for you to review off-line. Other participants observing the audit may also be responsible for collecting those documents in real-time to ensure the audit can continue without any delay. Finally, content in the chatbox can be recorded as a text file automatically.
  • Tour Guide – Video chat allows auditees to bring auditors into physical places of their facility as if the auditor were there in person. Production employees can be interviewed, in person and in real-time, while the employee demonstrates processes. You can show how nonconforming materials are labeled and segregated to keep them from accidentally being used for production. When requesting this audit method in an audit agenda, the lead auditor should recommend a dedicated “camera person” with a mobile phone and selfie stick, because it is challenging to answer auditor questions and operate a video camera simultaneously. Remember, remote audit resources consist of hardware, software, and people. 

My favorite remote auditing tools (hardware)

My favorite hardware resource is the Pixelbook that I am using to write this article. We write audit reports with Google Docs instead of Microsoft Word because multiple team members can simultaneously edit the same document without creating conflicting versions. We operate Zoom video web conferencing software to speak with auditees and clients, but we use the Pixelbook to type our notes and audit reports. The Pixelbook is lightning fast, and it is a little smaller, so there is just enough room on your desk next to a laptop. The most significant advantage of using Google Docs is realized when you are the lead auditor of an audit team. As a lead auditor, you can type notes in the section of the audit report that other team members are working on, to make sure that they include audit trails from other members of the audit team. This is also an extremely useful technique when you are training a new auditor, and you want to guide them without disrupting the flow of an interview with a subject matter expert.

My second favorite hardware resource is an HD webcam mounted on a flexible arm with a clamp (see picture above). The video quality is 1080p instead of the 720p that is typical of a laptop camera. The flexible arm is equally essential because you can look directly at the camera while I’m simultaneously looking at the monitor. The only thing I dislike about the webcam I am using is the audio quality. Therefore, I use a gaming headset with a microphone to record the audio, so I can hear the people I am interviewing better. Another alternative is high-quality microphone and headphones, as typically seen in use by podcasters. Even though the sound quality is ideal with a separate microphone and headphones, the cost is higher than most gaming headsets, and you will be tethered to microphone–either physically or at least virtually by the need to maintain a consistent distance between your mouth and the microphone. The more hours you spend at the computer, the more you will appreciate the ability to stand up, adjust the camera, and move your legs a little.

Finally, the last piece of essential remote auditing hardware is your mobile phone. Even with a desktop running Zoom, and a Pixelbook running Google Docs, I still need to ask audit team members questions and conduct quick internet searches. Therefore, your mobile phone is essential to keep with you, in silent mode, during your audit. If you don’t have your phone, then you need to stop sharing your screen and send a message during your audit. Your phone is much less disruptive. I use the phone to keep track of time, to set reminder alarms, and to send Slack messages with other people. You can also join a separate Zoom session on your phone, where an audit team member may need you (the lead auditor) to provide input on objective evidence or evaluation of conformity regarding specific quality system requirements. You might also want to take a quick picture of something you observe on video during the audit. If you record the Zoom session, you can always extract a still image, but taking a picture with your mobile phone is more convenient and takes less time. You can then share the image with a Google Drive folder for your remote audit and copy the image into your audit report. As they say, a picture is worth 1,000 words.

One last note on hardware: a 48” flat screen is great for virtual bike rides on your trainer (as seen in the picture above), but it’s just a little too big for a desktop monitor. It’s excellent for side-by-side viewing, but dual monitors are a better approach.

Remote Auditing Resources for Web Conferencing

Currently, we are using Zoom as our video web conferencing software. Still, we used to use GoToMeeting, and there is very little difference in the functionality of the two software platforms. One of the consequences of the COVID19 pandemic is that everyone is more familiar with web conferencing software. Here are a few other options you could consider, including Slack, which we use as a messaging tool, and we have integrated with Zoom within our team’s channel.

  1. Google Meeting
  2. Skype
  3. Microsoft Teams
  4. Monday
  5. Slack

Remote Audit Resources for Scheduling Your Audit

Currently, we are using Calendly as the automated appointment scheduling software application for our consulting business. However, the functionality of software applications has changed dramatically in the past few years with better integration tools, such as Zappier.  Therefore, don’t be surprised if we change to one of the applications listed below. These applications allow you to manage people, equipment, and conference rooms, but you can also integrate these applications with accounting business processes.

  1. Simplybook.me 
  2. Acuity Scheduling 
  3. Jobber 
  4. Gigabook
  5. vcita

Remote Auditing Accessories

We hosted three international training workshops, and we record training videos for medical device companies every week. Therefore, we gradually accumulated all of the accessories listed below. Technology gadgets for recording videos are continually changing, and our best advice is to save your money. Instead, rely upon a mobile phone and an extra person with “the original selfie sticks” (i.e., arms). Once you complete your first remote audit, then you can think about which of the latest gadgets might make your life easier.

  • Selfie Sticks
  • Tripod
  • External microphones
  • Portable Batteries
  • Additional lighting

If you have any suggestions for additional hardware and software for remote auditing, please add a comment to this article so we can keep this up to date with the latest technology. 

Future Articles & Webinars

Thank you for reading. This article is our third in a ten-part blog series specific to remote auditing techniques:

  1. Remote audit opening meeting – 4 changes – May 12
  2. Audit team communications – May 19
  3. Remote audit resources – software and hardware tools – May 26
  4. How to apply a risk-based auditing approach to audits and remote audits – June 2
  5. How to make a supplier questionnaire for remote auditing – June 25
  6. Remote audit duration less than 90 minutes – June 30
  7. Remote auditing work instruction – July 14
  8. Planning partial remote audits – July 21
  9. Remote audit invitations – 4 things to remember – August 4
  10. Training new audit team members and lead auditors – August 11

Five (5) new webinars planned on related topics:

  1. Opening Meetings Webinar (free) – May 14, 2020
  2. Audit team communication during a remote audit (free) – June 4, 2020
  3. How to qualify your suppliers webinar (pre-order by June 1) – June 25, 2020
  4. Remote auditing techniques webinar (pre-order by July 1) – July 16, 2020
  5. MDSAP Certification Body Interviews (free) – August 6, 2020

Remote audit resources – software and hardware tools Read More »

Audit team communication

4 Comments / Remote Auditing / By

The lack of visual cues may hinder communication between the auditor and the auditee, but software tools can enhance audit team communication.

Audit Team Communication Cup Phone Audit team communication

Audit Team Communication Requirements

During the opening meeting, the lead auditor is responsible for confirming the “formal communication channels between the audit team and the auditee…[and] the auditee being kept informed of audit progress during the audit” (ISO 19011:2018, Clause 6.4.3). Typically, the audit program manager will follow the lead auditor during the audit. In that situation, audit team communication with the auditee is direct and verbal. However, if the audit team consists of multiple auditors, the lead auditor also needs to establish a method of communication between the team members and the lead auditor. Team members need to make the lead auditor aware of any potential nonconformities, but more critical information includes:

  1. audit trails that require follow-up by auditors in other process areas
  2. any delay experienced by team members
  3. if an audit team member is ahead of schedule

Communication Limitations During On-Site Audits

During an on-site audit, it is not uncommon to have limited communication with the rest of the team, because the team is interviewing auditees and walking through the facility–not sitting at their computer. Sometimes your cellular signal is inadequate for texting or other messenger services such as Slack. It may also be more difficult to have private discussions between team members during an on-site audit. Usually, the audit schedule is very tight, and team discussions must wait until lunch breaks or scheduled team discussions. Unfortunately, these limitations frequently result in the follow-up of audit trails waiting until the very end of the audit, instead of addressing audit trails at more convenient times in the middle of the audit.

Communication Between Auditors During Remote Audits

During a remote audit, all of the audit team members will readily be able to exchange information by email, text, or Slack. Besides, applications like Google Docs allow multiple auditors to type in the same audit report simultaneously. Therefore, auditors can type a specific follow-up item in the section of the audit report, where another auditor will be typing their notes for the applicable audit area. For example, if one auditor is interviewing incoming inspection activities, they can type a note for the auditor that will be auditing calibration to review the calibration certificates for inspection devices used in the incoming inspection process. If an audit team leader needs more time, they can type a quick note for the lead auditor about the need for more time. The lead auditor can also quickly send a Slack message to the rest of the audit team, asking if anyone can aid the audit team member that is behind schedule. This communication is efficient, documented directly within the report, and occurs in real-time. The result is that communication between team members is more effective, and the audit is completed earlier.

Improvement of Auditor Training with Remote Auditing

When audit team members are being trained, the lead auditor must observe their auditing and provide constructive feedback. Ideally, the lead auditor will wait for a “teachable moment.” This is the moment immediately after the lead auditor-in-training makes a mistake. Telling an auditor-in-training what to do during an audit teaches the auditor little. However, if the auditor is allowed to make a mistake, such as forgetting to ask for an audit record, then the lead auditor can point out the error immediately afterward. Correcting the auditor can be as simple as adding a note in red font within the audit report in the same section where the auditor is currently typing. The auditor will see the comment and make the correction, but the auditee will not be aware of the error. This approach avoids any embarrassment to the auditor, and the auditor is more likely to remember the instruction as constructive feedback that will make them better.

Remote Auditing Can Be Easily Recorded

Auditors can learn from the constructive feedback provided by a lead auditor, but they can also learn by watching and listening to themselves if the remote audit is recorded. This is especially easy to accomplish for internal audits, but suppliers may also allow recording of certain process audits. Opening meetings, closing meetings, and common procedures such as incoming inspection usually do not include confidential information. Therefore, you should be able to obtain permission to record these portions of the audit. These recordings can be reviewed by the auditor to identify when poorly worded questions were used. Auditors-in-training can identify when they miss an opportunity to follow an audit trail, or an auditor may realize that they ask auditees certain closed-ended (i.e., yes/no) questions instead of open-ended questions that will help them gather more information from the auditee.

Audit Team Communication with Guides

In addition to the communication between the lead auditor and the audit team members, audit team members also need to communicate with their audit guides. Guides should be used to communicate messages throughout the company. For example, if the audit is behind or ahead of schedule, the guide can communicate adjustments in the timing of the agenda. If an audit team member requests records to be provided, the guide can communicate this request and make sure the records are waiting for the auditor when they return to the audit conference room. Guides also are responsible for helping the audit team navigate from one process area to another during the audit, and to make sure that the audit team observes all safety and gowning requirements during the audit. Finally, guides may also be asked to act as an observer and verify objective evidence collected by the auditor.

Shifting Role of a Guide During Remote Audits

During a remote audit, requests for records to be provided and communication of deviations from the agenda can easily be communicated by the auditor chat features in the video conference, instant messengers, or email. Therefore, you might think that a guide is unneeded. However, when audit team members request viewing another area of a facility during a remote audit, it may be necessary to provide live video images of the process areas. It isn’t easy to speak with the auditor and provide live video images. It may be dangerous to walk backward through your facility, carrying a selfie stick, and concentrating on your discussion with the auditor instead of where you are walking. Instead, the guide should focus on providing live video, and the process owner should be concentrating on providing a guided tour and answering the auditor’s questions. The guide may also be asked to record certain information in video or picture format as objective evidence.

Conclusion

Audit teams should practice using shared documents in Google Docs and Slack during the audit to facilitate real-time audit team communication. Google Docs enables everyone to write their audit notes directly into an audit report template to eliminate delays in the completion of the audit report. Using Google Docs also makes it possible for the lead auditor to observe the progress of the audit in real-time. Audit team communications of audit trails for team members to follow-up can be accomplished in real-time by just adding a note about the trail in the applicable section of the audit report. Finally, remote auditing can facilitate better training of auditors.

Audit team communication Read More »

Remote audit opening meeting – 5 changes

2 Comments / Remote Auditing / By

This article describes five minor adjustments that lead auditors should make when they plan a remote audit opening meeting.

Remote Audit Opening Meeting Remote audit opening meeting 5 changes

Regardless of whether you are conducting an on-site audit or a remote audit, the first activity conducted during the audit is an opening meeting. The process for conducting opening meetings is defined in ISO 19011:2018, Clause 6.4.3, and it is the responsibility of the lead auditor to lead this meeting. There are three purposes to the opening meeting:

  1. confirm agreement to the audit plan,
  2. introduce the roles of the audit team, and
  3. ensure the audit can be conducted as planned.

Opening meeting checklists

There is a long list of items that are typically confirmed during the opening meeting. New auditors are trained to rely upon an opening meeting checklist to ensure that none of the things on the list are accidentally forgotten. Some auditors will depend on a formal presentation during an opening meeting, but usually, this requires more time to set-up. Therefore, most auditors work from a pre-written checklist on their computer or paper.

Change #1: Presentations replace checklists in a remote audit opening meeting

If you are conducting a remote opening meeting, most of the attendees will be looking at a computer screen. The lead auditor can share their screen as they go through a formal presentation, without wasting any set-up time during the opening meeting. Also, attendees can be emailed the presentation before the opening meeting, along with the audit plan. If you are the lead auditor planning a remote audit, you should use an opening meeting presentation template to make sure that none of the items in clause 6.4.3 are skipped. If your company is developing a work instruction for conducting audits remotely, you should create a controlled template to ensure consistency among auditors. This should also be done for closing meetings. You can learn more about conducting opening and closing meetings in our webinar on May 14, 2020.

Change #2: Every audit team member should create a personal slide

One of the challenges of being remote is that you have trouble establishing rapport with the auditees. To overcome this challenge, you should use live video to show your face, smile, and say hello to auditees. It would help if you also created a slide for the opening meeting presentation that includes a personal picture that conveys your congenial character and less formality. You should also include your preferred method(s) of contact during the audit, such as email address, mobile phone number, or Slack @username. If you are part of a team, you should also present the slide(s) that explain which process areas you will be responsible for auditing. If you have any special needs, such as vision or hearing impairment, you should also indicate how you prefer auditees to communicate with you.

Change #3: Edit the agenda during the remote audit opening meeting

Auditors confirm the planned agenda with the auditees during the opening meeting, but adjustments to the schedule are typical of remote and on-site audits. At most on-site opening meetings, everyone will have a hardcopy of the agenda and make notes on their agenda to reflect schedule changes. It is the responsibility of the lead auditor to distribute an updated version of the revised agenda and to include the updated agenda with the audit report. However, in a remote audit opening meeting, the lead auditor should share a copy of the agenda with everyone in a software tool like Google Docs(see below). When changes are made, switch screen sharing from your presentation to the agenda. You can make the changes in view of all attendees. Also, if you share the document with auditees, they can correct errors in the audit agenda for you (e.g., the spelling of names) and often with greater efficiency than giving you a verbal explanation of the changes.

Google Docs Audit Report Template Remote audit opening meeting 5 changes

Change #4: Verify meeting invitations are updated at the end of the opening meeting

When there is an audit team conducting a remote audit, each auditor should send out a separate meeting invitation and include the lead auditor. This is important because each of the auditors needs to be able to audit simultaneously, but they may need the lead auditor to join their segment of the audit briefly. When changes are made to the audit agenda, such as changing the sequence of process areas being sampled, the time of the invitations needs to be updated for everyone involved. The lead auditor should verify that all of the invitations on their calendar match the updated agenda.

Change #5: Record your remote audit opening meetings (and closing meetings)

Recordings document critical information that might not be captured in the notes of the lead auditor while they are presenting. Therefore, requesting permission to record an opening and closing meeting of an audit is recommended. More importantly, if anyone is absent, the recording can be shared with that person. Finally, recordings allow you to “replay” mistakes and successes. The ability to replay the meeting, and observe yourself, is an invaluable tool for lead auditors in training and anyone that wants to improve.  

How long should your opening meeting be?

Audits are challenging to complete on-time, and therefore shorter opening meetings are desirable. However, the opening meeting is also dependent upon the scope of activities being audited and the number of audit team members. A duration of 30 minutes is typical for an on-site audit, but the opening meetings are often preceded by casual discussion and informal greetings. Teleconference calls and video chat meetings are less conducive to informal greetings because it is difficult for two people to speak at the same time. The remote meetings also seem more likely to start on-time. Therefore, you should expect a remote audit opening meeting to be more efficient (i.e., shorter).

Remote audit opening meeting – 5 changes Read More »

What’s the difference between PMS, PSUR, and PSR?

4 Comments / Post-Market Surveillance / By

This blog is intended to help clear your justified confusion if you are wondering what the difference is between PMS, PSUR, and PSR.

 

The nine most terrifying words in the English language are, “I’m from the Government, and I’m here to help.” That quote is from a speech by President Reagan on August 12, 1986.  One of the goals of the European Parliament and Council was “to ensure effective coordination of [competent authority] market surveillance activities and to clarify the applicable procedures.” After studying the new European MDR, I can confidently say that the European Parliament and Council have done their job well. My boss is a regulatory consultant with 30 years of experience, and he asked me to explain the difference between PMS, PSUR, and PSR.

To answer that question as objectively as possible, and cite my sources, I have included a copy and paste directly from Regulation (EU) 2017/745. Red text is my commentary, while the italicized text is a quotation from the most relevant article within the new EU regulations.

Under the New MDR, the only Class IIa, Class IIb, and Class III products are definitively required to have a Periodic Safety Update Report (PSUR). The PSUR needs to be updated annually for Class III and Class IIb implants, and the PSUR needs to be updated at least every two years for Class IIb (non-implants) and Class IIa devices. The PSUR must be available to your notified body, and upon request, the competent authorities. In contrast with the PSUR, Post-Market Surveillance (PMS) reports are required for Class I devices. Finally, a manufacturer’s Periodic Summary Report (PSR), relates to specific cases of Serious Incidents and Field Safety Corrective Actions (FSCA’s) based upon an agreement between the manufacturer and the competent authority or authorities instead of submitting individual FSCA reports.  This is confusing because the PSUR also meets the requirements of a PMS Report as defined in Article 85, but we don’t call it a PMS Report.

“Article 83 – Post-market surveillance system of the manufacturer

1. For each device, manufacturers shall plan, establish, document, implement, maintain, and update a post-market surveillance system in a manner that is proportionate to the risk class and appropriate for the type of device. That system shall be an integral part of the manufacturer’s quality management system referred to in Article 10(9).”

In Matthew’s words, “Manufacturers are required to establish a PMS system for every device or device family.”

“Article 84 – Post-market surveillance plan

The post-market surveillance system referred to in Article 83 shall be based on a post-market surveillance plan, the requirements for which are set out in Section 1.1 of Annex III. For devices other than custom-made devices, the post-market surveillance plan shall be part of the technical documentation specified in Annex II.”

In Matthew’s words, “Article 84 requires you to have a PMS plan in your quality system.”

“Article 85 – Post-market surveillance report

Manufacturers of class I devices shall prepare a post-market surveillance report summarizing the results and conclusions of the analyses of the post-market surveillance data gathered as a result of the post-market surveillance plan referred to in Article 84 together with a rationale and description of any preventive and corrective actions taken. The report shall be updated when necessary and made available to the competent authority upon request.”

In Matthew’s words, “A Class I device requires a PMS report, while the other product classifications require a PSUR.”

“Article 86 – Periodic safety update report

1.1 – Manufacturers of class IIa, class IIb, and class III devices shall prepare a periodic safety update report (‘PSUR’) for each device and were relevant for each category or group of devices summarizing the results and conclusions of the analyses of the post-market surveillance data gathered as a result of the post-market surveillance plan referred to in Article 84 together with a rationale and description of any preventive and corrective actions taken. Throughout the lifetime of the device concerned, that PSUR shall set out:

(a)

the conclusions of the benefit-risk determination;

(b)

the main findings of the PMCF; and

(c)

the volume of sales of the device and an estimated evaluation of the size and other characteristics of the population using the device and, where practicable, the usage frequency of the device.

Manufacturers of class IIb and class III devices shall update the PSUR at least annually. That PSUR shall, except in the case of custom-made devices, be part of the technical documentation as specified in Annexes II and III.

Manufacturers of class IIa devices shall update the PSUR when necessary and at least every two years. That PSUR shall, except in the case of custom-made devices, be part of the technical documentation as specified in Annexes II and III.

For custom-made devices, the PSUR shall be part of the documentation referred to in Section 2 of Annex XIII.

  1. For class III devices or implantable devices, manufacturers shall submit PSURs by means of the electronic system referred to in Article 92 to the notified body involved in the conformity assessment in accordance with Article 52. The notified body shall review the report and add its evaluation to that electronic system with details of any action taken. Such PSURs and the evaluation by the notified body shall be made available to competent authorities through that electronic system.
  2. For devices other than those referred to in paragraph 2, manufacturers shall make PSURs available to the notified body involved in the conformity assessment and, upon request, to competent authorities.”

In Matthew’s words, “Barring specified exemptions, manufacturers of a Class IIa device would need to submit a PSUR and update it at least every two years.”

“Article 87 – Reporting of serious incidents and field safety corrective actions

9. For similar serious incidents that occur with the same device or device type and for which the root cause has been identified or a field safety corrective action implemented or where the incidents are common and well documented, the manufacturer may provide periodic summary reports instead of individual serious incident reports, on condition that the coordinating competent authority referred to in Article 89(9), in consultation with the competent authorities referred to in point (a) of Article 92(8), has agreed with the manufacturer on the format, content, and frequency of the periodic summary reporting. Where a single competent authority is referred to in points (a) and (b) of Article 92(8), the manufacturer may provide periodic summary reports following an agreement with that competent authority.”

In Matthew’s words, “Periodic summary reports (PSRs) refer to significant incidents (SIs) and field safety corrective actions (FSCAs). PSRs require an agreement between the manufacturer and the competent authority(s) for cases where there is a group of common, well-known, and documented SIs or FSCA’s with a known root-cause. PSRs are an alternative to submitting individual SI and FSCA reports.”

Additional Quality System Resources

My boss also asked me to update the procedures for post-market surveillance (SYS-019) and vigilance (SYS-036). The PMS procedure includes requirements for Articles 83-86. The vigilance procedure includes the requirements for Articles 87-92.

About the author

20190531 005146 150x150 Whats the difference between PMS, PSUR, and PSR?

Matthew is a talented writer that missed his calling as a political satirist. Medical Device Academy is lucky to have him as a quality system expert and gap analysis guru. Matthew was asked to answer this question for a client in response to an email. He wrote the entire blog in less than one hour, but he didn’t think it was worthy of publishing. The boss disagreed. Please show Matthew some love with your comments below or by ordering the book from Amazon ($5 pre-order discount until August 28, 2020).

What’s the difference between PMS, PSUR, and PSR? Read More »

Changes triggered by COVID19 in your quality system

3 Comments / Business Continuity Plan, Quality Management System, Remote Auditing / By

The 2020 global pandemic has changed life as we know it, but this article focuses on three crucial quality system changes triggered by COVID19.

3 things COVID19 changed 2 Changes triggered by COVID19 in your quality system

Last night my daughter Gracie mentioned that her teacher assigned an essay to write about three changes triggered by COVID19 in her life. The three things that she felt had changed the most were: 1) she goes to bed much later, and sleeps in every day; 2) her school is closed, and she only talks to her teacher twice per week via Zoom, and 3) she misses her friends. I know that her story is similar to my son Bailey who is in his Freshman year of college, and I know that my personal story is quite similar. Coincidentally, I started writing this article earlier this week about three significant-quality system changes triggered by COVID19:

  1. If you are going to conduct on-site audits, you need to ask about using personal protective equipment (PPE).
  2. There needs to be a greater focus on business continuity plans and robust supply chain monitoring.
  3. Remote audits are suddenly encouraged for 1st, 2nd, and 3rd-party audits.

Changes triggered by COVID19: #1 Use Face Masks

US FDA Issues EUAs

At the beginning of the COVID19 pandemic, the US FDA created several emergency use authorizations (EUA). The three EUA areas were IVD testing, ventilators, and face masks. The EUA for IVD testing is not surprising, because the FDA issues and EUA every time a new lethal and contagious virus emerges (e.g., Zika and Ebola). The EUA for ventilators was issued because the number of people with respiratory issues was expected to explode with the spread of the virus, and the supply chain for components of ventilators had already been disrupted by the initial spread of the virus in China. The EUA for face masks was issued because it is the second-best way to protect people from the virus, and existing infrastructure for face mask production could not possibly supply the entire world with face masks overnight.

Everyone in the World Gets a Face Mask

As soon as the EUA for face masks was issued, every regulatory consultant in the USA was inundated with urgent requests for help to complete EUA requests for masks. I also received similar requests for assistance with Canadian filings. The FDA did a great job of providing detailed information about the different types of face masks (i.e., face masks, surgical face masks, and N95 respirators). Testing companies created new website pages specifically for each of the different face mask tests, and every company with a sewing machine suddenly wanted to manufacture masks. I even read an article about an elderly woman making face masks for her entire family while she listened to The Beatles “HELP!” in the background.

Why aren’t you wearing your face mask?

Even after the world makes the first 7 billion face masks, not everyone will wear their face masks. Masks will protect us from touching our hands to our face–which spreads many germs in addition to the SARS-CoV-2 virus. Masks will also keep us from coughing on other objects and people if we have the virus. Finally, face masks protect us from the small droplets that carry the virus from one person to the next. Even though there are obvious safety reasons for everyone in the world to wear a face mask, most people don’t want to wear a face mask. This is no different from the argument to wear a seat belt, and unless our government creates a law or temporary order requiring us to wear face masks, most people won’t bother to wear one.

Changes triggered by COVID19: Auditors need to wear face masks 

As a medical device auditor, I feel I must always follow the safety rules in every facility I visit. Lead auditors are supposed to contact the company ahead of time and ask about the safety policies as part of audit preparation and initiating the audit. I’m 6’6” (2.00m) in height, and my shoe size is 14. There is seldom gowning for me to wear that fits appropriately–especially in Southeast Asia. I squeeze into the garments, and they are uncomfortable and hot, but I wear the garments anyway. My job includes auditing clean rooms, and I can’t do my job without gowning up. By following the rules, I also eliminate the excuses for anyone in the facility I visit. Now that we have a global pandemic, you should be wearing a face mask in every medical device facility to protect yourself, people you work with, and users of medical devices. You should also consider carrying spare face masks with you to protect yourself on airplanes, in hotels, etc.

Changes triggered by COVID19: #2 Business Continuity Plans

Will business continuity plans be required now?

In addition to the cultural shift to wearing face masks, we will also need to make significant changes in our overall preparations for natural disasters, fires, and biological threats. Although there is no specific requirement for a business continuity plan in ISO 13485:2016, there are many places where an auditor can identify a requirement to maintain the effectiveness of a quality system (no exceptions):

  1. Clause 1, Scope
  2. Clause 4.1.1 & 4.1.3, General Quality System Requirements
  3. Clause 5.3, Quality Policy
  4. Clause 5.4.2, Quality management system planning
  5. Clause 5.6.3, Management Review Output
  6. Clause 6.1, Provision of resources
  7. Clause 8.1, General requirements for Measurement, analysis, and improvement
  8. Clause 8.2.4, Internal audit
  9. Clause 8.5.1, General Improvement

Although any of these clauses could potentially be referenced as a requirement for a business continuity plan, the last clause would generally be the most appropriate. This clause states, “The organization shall identify and implement any changes necessary to ensure and maintain the continued suitability, adequacy and effectiveness of the quality management system…”. In this time of radical change, adding provisions to your business continuity plan for coping with a global biological threat seems obvious and urgently needed.

Suggested content for your business continuity plan

Sadly, the USA was probably better prepared for a disaster in the 1960s after the Cuban Missile Crisis than we are today. If you do not yet have a business continuity plan, or if you need suggestions for improving your plan, the following is a list of suggested items to include in your plan:

  1. Develop a plan for power outages, fires, floods, earthquakes, severe wind/tornadoes, hurricanes, workplace violence, and biological threats 
  2. Develop an emergency alert system to notify employees of any emergency
  3. Build emergency kits and store the kits for when they are needed
  4. Document your plan in multiple formats (virtual and physical) and distribute to all employees–including a social media plan
  5. Translate your plan into multiple languages for non-English speaking employees
  6. Develop a training program that addresses the various aspect of emergency preparation
  7. Practice your plan just like fire drills, so everyone is prepared and nobody panics

The Ready.gov website has many resources for the above items, including a series of “Ready Business Videos” and “Ready Business Toolkits.”

How to practice your business continuity plans

My sister is a teacher, and she is in the process of opening a new charter school in Maine. We were discussing her planning for the school, and the disruption of schools by the COVID19 pandemic has challenged all teachers to learn to use distance learning. My sister’s school focuses on teaching children about the environment, and she doesn’t like to spend lots of time on the computer. I was sharing some of the environmental studies my daughters are receiving via Zoom from their teachers. I suggested that she might want to pick one topic each week to teach via distance learning. The purpose of this would be to give her, and her students practice using distance learning for a variety of subjects. Therefore, when we experience another biological disaster, her students will already know precisely how to use distance learning to continue their education. My argument was that this routine use of distance learning would be a more effective preparation for emergencies than a once-per-month “fire drill.” Companies should use the same approach. Your company should create a schedule for practicing remote management meetings and working from home. This will ensure that systems are in place to keep your business running smoothly when disaster strikes again.

Changes triggered by COVID19: Expect regulators to require business continuity plans

The widespread shortage of face masks, ventilators, and other critical supplies needed during the COVID19 pandemic is going to result in new regulations requiring business continuity plans. This is a certainty born from the observation that every single medical device regulation we have resulted from severe public health threats. The COVID19 pandemic is the biggest global health crisis the world has experienced in 100 years. Therefore, we can expect corrective actions in the form of new regulations requiring companies to have a business continuity plan. Some regulators will act independently, but I would expect this also to be an action taken by the International Medical Device Regulators Forum (IMDRF). We can also expect there to be new laws requiring amendments to business continuity plans for public companies. The Sarbanes-Oxley Act of 2002 requires public companies in the USA to have business continuity plans. Despite this requirement, many public companies have been ruined by the COVID19 pandemic. Therefore, we should expect amendments to these requirements and revisions to the international standard for business continuity planning (i.e., ISO 22301:2019). We should also expect to see new interest in becoming certified to this standard.

Changes triggered by COVID19: #3 Remote Auditing

What are certification bodies doing about surveillance audits and re-certification audits?

Most of the companies that had initial certification audits scheduled for the first quarter of 2020 were forced to reschedule their audits because the employees must work from home, and the certification bodies must conduct at least some of their audits on-site. The FDA was also forced to cancel all foreign inspections temporarily. However, companies that already have certification need surveillance audits and re-certification audits to maintain the validity of their quality system certificates. Therefore, certification bodies now have plans for conducting audits remotely. For companies that virtual medical device manufacturers, certification bodies can conduct full quality system audits remotely. However, manufacturers with production activities on-site are only able to conduct partial audits. The certification bodies must still conduct on-site audits, but they are being permitted six months to conduct an on-site audit to cover the gaps remaining from the partial remote audits. Prior to conducting the partial remote audits, certification bodies are sending out questionnaires to all of their clients to gather information about whether the manufacturers can support a remote audit and to what degree.

Second-party audits conducted remotely

Second-party audits, also known as supplier audits, have always been of interest for manufacturers to conduct remotely–mainly if the supplier is located overseas. The US FDA regulations do not require companies to conduct supplier audits. However, if there are quality problems with suppliers, you are expected to conduct a thorough investigation to identify the root cause of quality problems, in most cases, that require an on-site audit. However, if your suppliers are providing good quality and they are ISO 13485:2016 certified, then you probably are using this as a justification for not conducting on-site audits or at least reducing the frequency of those audits. Now that most people are not able to travel, or because the people you need to speak with are working from home, manufacturers are being forced to conduct remote audits. This has always been permitted, but the effectiveness of remote audits is often questioned. Supply chain disruptions are now a global issue that is impacting the safety and effectiveness of our hospitals, and regulators will expect you to improve the rigor of your supplier evaluations–including conducting more supplier audits. Therefore, establishing more effective procedures for remote supplier auditing is urgently needed.

Changes triggered by COVID19: We need to develop procedures for remote auditing

Although most first-party audits are conducted on-site, especially if conducted by employees of your company, we will still need to establish procedures for remote auditing for internal audits. Some of our client’s scheduled internal audits for April and May that they had to cancel because they were unable to access the records needed for the audit while they were working from home. In addition, most of the US States have implemented stay-at-home audits that prevent our team from traveling to our clients. This is forcing our team to develop more robust procedures for remote auditing. We needed to change our audit agendas to accommodate eight 90-minute audit sessions in four days, instead of conducting two full days of on-site auditing. We are also doing more preparation before the audit to allow the auditees time to scan paper records so that we can review those records remotely. Finally, we are experimenting with techniques for collaboration as an audit team so that multiple auditors can simultaneously audit a client and complete a full quality system audit more quickly without forcing any one person to work for longer than 90 minutes in front of a computer. We are still perfecting these new methods, but we are writing a series of articles on this topic. You can order the book from Amazon ($5 pre-order discount until August 28, 2020).

Thank you & Future Articles

Thank you for reading. This is the longest article we have published on our site since 2012. This article also kicks off a ten-part blog series specific to remote auditing techniques:

  1. Remote audit opening meeting – 4 changes – May 12
  2. Audit team communications – May 19
  3. Remote audit resources – software and hardware tools – May 26
  4. How to apply a risk-based auditing approach to audits and remote audits – June 2
  5. How to make a supplier questionnaire for remote auditing – June 25
  6. Remote audit duration less than 90 minutes – June 30
  7. Remote auditing work instruction – July 14
  8. Planning partial remote audits – July 21
  9. Remote audit invitations – 4 things to remember – August 4
  10. Training new audit team members and lead auditors – August 11

There are also five new live webinars planned on related topics:

  1. Opening Meetings Webinar (free) – May 14, 2020
  2. Audit team communication during a remote audit (free) – June 4, 2020
  3. How to qualify your supplier’s Webinar (pre-order by June 1) – June 25, 2020
  4. Remote auditing techniques webinar (pre-order by July 1) – July 16, 2020
  5. MDSAP Certification Body Interviews (free) – August 6, 2020

Changes triggered by COVID19 in your quality system Read More »

NSE letter: A CAPA plan for your 510k process

1 Comment / 510(k), CAPA, Design Control / By

Cry, complain, call the reviewer…you might feel a little better, but you received an NSE letter, and tomorrow you still can’t sell your device.

NSE Letter NSE letter: A CAPA plan for your 510k process

Instead, try approaching an NSE letter like a CAPA investigation. What is the issue? The FDA determined that your device is not substantially equivalent to the predicate you selected. What is the root cause? There are four (4) possible root causes.

NSE Letter Cause #1: You failed to verify that the predicate is a legally marketed device.

If your predicate device is not legally marketed, you need to select a new predicate and resubmit. However, it is doubtful that your device would pass the refusal to accept the (RTA) screening process if the predicate was not legally marketed. If your predicate was not registered and listed with the FDA (check using this link), then you should have submitted a pre-sub request to determine if the agency has any problem with using the device you chose as a predicate. This is an essential question if the manufacturer is no longer in business, and the product is no longer for sale.

NSE Letter Cause #2: You failed to evaluate the substantial equivalence of your device’s intended use with the predicate.

The intended use of your predicate device is documented for every potential predicate since February 1992 on FDA Form 3881–which you can download along with the 510k clearance letter for the predicate. There is also an intended use documented for every device category in the applicable regulation for that device. This intended use is more generic than FDA Form 3881, but both are applicable. The FDA Form 3881 you submit for your device must be equivalent. I recommend a point-by-point comparison with regard to the following elements: 1) OTC vs. prescription use, 2) user, 3) patient population, 4) illness or medical condition, 5) duration of use, 6) environment of use and 7) target part of the body. Any difference can raise new issues of risk and may result in an NSE decision. However, the FDA typically will work with the company to modify the wording of FDA Form 3881 to ensure the intended use is equivalent or to make sure you provide clinical evidence to address the differences. In my pre-submission requests, I include a comparison document for the intended use to ensure that the FDA is aware of any differences in the intended use.

Cause #3: You failed to convince the FDA that technological differences do not raise different questions of safety and effectiveness.

Unless your device is identical in every way to the predicate device, you will have to persuade the FDA that differences do not raise questions of safety and effectiveness. At the beginning of the 510(k) process, it is helpful to document technological differences systematically. Specifically, this should include: 1) materials, 2) design, 3) energy source, and 4) other features. For each difference, you must justify why the difference does not raise different issues, or you must provide data to prove it. It is also possible that you were not aware of questions of safety and performance raised by technological differences. To avoid this problem, you can submit a detailed device description and draft labeling to the FDA in a pre-sub meeting request. If you ask questions about differences in a pre-sub meeting, you can avoid an NSE letter.

Cause #4: You failed to provide data demonstrating equivalence.

For each difference, you should determine an objective method for demonstrating that the difference is equivalent in safety and performance to the predicate. Your test method can be proposed to the FDA in a pre-sub request before testing. The FDA sees more than 3,000 companies propose testing methods to demonstrate equivalence each year. They have more experience than you do. Ask them in a pre-sub before you test anything. There may be a better test method, or you might need to adjust your test method. Sometimes results are unclear, but there might be another test you can perform to demonstrate equivalence, and then you can resubmit your 510k. Possibly you were unaware of the need to perform a test, and you were unable to complete a test within the 180 days the FDA allowed for submitting additional information. The good news is you now have all the time you need.

What is similar between all four causes of the NSE letter?

In all four root causes identified above, you could benefit greatly from the pre-sub meeting. Now you have an NSE letter, and you know which of the four reasons why your submission did not result in 510(k) clearance. However, the correction to your NSE letter may not be clear. Therefore, you should consider requesting a pre-sub meeting as quickly as you can. Most companies choose not to submit a pre-sub meeting request because they don’t want to wait 60-75 days. However, sometimes pre-sub meetings are scheduled sooner. In addition, 60-75 days is not as costly as receiving a second NSE letter.

Prevent a future NSE letter by requesting a pre-sub meeting

Regardless of your corrections for the current NSE letter, you should prevent future occurrences by planning to submit a pre-sub meeting request for every submission. I try to help clients gather all the information they need without a pre-sub meeting, but each new 510k reminds me why a pre-sub meeting is so valuable. You always learn something that helps you with the preparation of your 510k.

Help with Pre-sub meeting requests

The FDA published a guidance document for pre-sub meeting requests. If you need additional help, there is a webinar on this topic.

NSE letter: A CAPA plan for your 510k process Read More »

Scroll to Top