Would you like to learn the five most common grant writing mistakes medical device companies make when seeking funding?
Why do most medical device companies fail grant writing?
Many medical device companies will face unique challenges in grant writing that can lead to failure. One common issue is a lack of alignment between the company’s objectives and the funding agency’s priorities, as grants often emphasize societal impact or public health benefits over market potential. Companies may also struggle with presenting their proposals effectively, using overly technical language that alienates reviewers or neglecting critical aspects like societal impact and feasibility. Limited resources, such as insufficient time, staff, or grant writing expertise, exacerbate the problem, as does the absence of strong collaborations with research institutions or public health organizations. Additionally, companies frequently underestimate development timelines, submit proposals prematurely, or fail to justify their budgets convincingly.
What are the most successful grants for device companies?
In the United States, prominent grant programs such as those offered by the NIH, NSF, and CDMRP play a significant role in supporting the design and development of medical devices. These agencies release funding opportunities multiple times a year, focusing on medical device innovations. The NIH allocates $1.3 billion annually to fund cutting-edge technologies. Similarly, the NSF receives approximately $190 million each year to support technological advancements across various industries, with a significant emphasis on medical devices, particularly those involving high-risk, high-reward technologies. Meanwhile, the CDMRP, through the Department of Defense, provides funding for medical devices with dual-use applications, targeting both military and commercial markets. These programs offer valuable resources for companies seeking to innovate in the medical device sector.
How often do grant applications get rejected?
The path to success is highly competitive, with a substantial number of proposals facing rejection. Understanding the acceptance rates, common pitfalls, and strategies for improvement can enhance the likelihood of securing SBIR funding.
Acceptance rates for SBIR proposals vary across agencies and phases:
Phase I: This initial phase focuses on establishing the technical merit and feasibility of the proposed research. Acceptance rates typically range from 10% to 20%, depending on the agency and specific solicitation. For instance, the National Institutes of Health (NIH) reported a success rate of approximately 25% for SBIR Phase I proposals in recent years.
Phase II: Building upon Phase I results, Phase II emphasizes the development of the proposed innovation. Acceptance rates are generally higher, around 40% to 50%, as applicants have already demonstrated initial feasibility. NIH data indicates a success rate of about 44% for SBIR Phase II proposals.
Common Mistakes
The secret to successful grant writing is not just better writing. Timely preparation, ignoring program officer feedback, and matching your technology with an appropriate grant are just a few of the secrets to better grant writing.
Lack of Timely Grant Preparation
The SBIR agencies typically have a specified window every year of when the solicitation topics will be released. Common mistakes companies make is that they do not know when the announcements are made and where to find them. Many companies who miss the announcements fail to prepare in a timely manner to submit a winning grant proposal.
Not Seeking Feedback from Program Officers
During each solicitation cycle, each agency offers support from their program officers. These program officers make their contact information readily available to the public. Program officers are your best friend when you have any questions in regards to the application process and specific topics. Prior to submitting the proposal, you can meet with the program officers to ensure your project is the right fit for the topic you are seeking to apply for.
Not Matching the Technology with the Best-Suited Agency
There are 11 different agencies that partake in the SBIR program with varying focus areas. There is some overlap between these agencies and companies who want to pursue SBIR grants for the first time can miss a better suited opportunity from other agencies. It is important to take all things into consideration from funding amounts, solicitation topics, and award rates. There may be a specific topic from an agency that has a high award rate. However, if your project does not meet the needs for the topic and agency, it does not guarantee that you will be awarded. Hence why speaking to a program officer can guide you to your best-suited topic.
Not Enough Focus on the Business Side
As part of your SBIR proposal there is a heavy emphasis on commercialization, many for Phase II applications. There are lots of companies who have a great team with higher education, amazing scientific breakthroughs. However, they lack the expertise on how to bring this product to the market. A large miss on companies who do not get awarded is due to the inability to communicate in the proposal on how they will bring their product to the market. Through the process, you have to keep in mind that your business plans are just as important to your science.
Not Complying with Solicitation Requirements
At the times of when the solicitations are released, you have access to the documents on the requirements. This includes the format of the proposal, key information that is required and detailed information on the innovation the agency is seeking for the topic. The reason these documents are provided is to be of your resource. Failure to comply with these requirements is almost guaranteeing a loss. The first step to success is reading through the solicitation documents thoroughly and asking questions to experts.
What are the secrets to successful grant writing?
If your company has already won a phase 1 SBIR grant, and you are applying for a phase 2 SBIR, make sure that you obtain written feedback from the US FDA on your regulatory strategy in apre-submission meeting request before your application deadline.
What can you do now to improve your chances of success?
Being awarded a SBIR award is never a guarantee but there are steps you can take to increase your likelihood of being awarded.
If you have never written an SBIR proposal, seek expert guidance. There are lots of intricacies and changes that happen for the SBIR program and especially within the agencies themselves. There are many experts who are paid to do the dirty work all day for you! These experts will be able to understand your science and needs for your business. Therefore, eliminating headaches for your company to spend time researching on how to best write a grant proposal, leaving time to focus on your innovative technology that will change society.
Start from Phase I. Many of the programs offer fast track programs to do a Phase I and Phase II proposal combined. As these are great opportunities to win larger grant funding amounts earlier on, your success is much lower. The fast track programs are a great opportunity if you can show you have the research completed that is needed when applying to a Phase I grant. Get your foot in the door through a Phase I award, win and then pursue a Phase II. This will increase your chances long term with the agency.
Have a strong team. Your company does not need a large team but an experienced one. Those on your team must be fluent in your technology and business. Even if you are just an individual, you must have the background to prove your strong understanding of your technology and business.
Do not stop trying! Most companies obtain success upon their second submission. Almost all agencies will provide scoring and/or feedback on your proposal. Use this feedback to enhance your next submissions increasing your likelihood of success.
As the Marketing Lead at BW&CO, Alexina holds a Bachelor of Business Administration in Marketing from the University of Houston. With a deep understanding of the grant funding landscape, she drives strategic marketing initiatives that elevate BW&CO’s brand and reach. Fluent in both English and French, Alexina excels at creating and executing campaigns that connect with diverse audiences. Her expertise includes developing content strategies, managing digital marketing efforts, and conducting market research to support business growth.
Learn three valuable tips for efficiently recording your inspection results in medical device manufacturing while remaining FDA-compliant.
What are the best ways to record inspection results?
If you are inspecting a lot of material at an incoming inspection, and the inspection plan calls for inspecting ten samples for length, what is the best way to record the inspection results?
The person who sent me this question also provided three options (read on for better suggestions):
Record the maximum and minimum dimensions
Record all ten measurements in a data collection table
Circle “pass” or “fail” next to each sample number
FDA requirements for incoming inspection results
The first method fails to meet the requirement as specified in 21 CFR 820.80(b) because recording only the maximum and minimum dimensions of the ten samples does not include the inspection results for the eight samples in between the extremes. The second method meets the requirements, but this method takes the most amount of time. The third method appears to meet the requirements. However, if you read the FDA requirements more carefully, 21 CFR 802.80(e)(3) states that “[Inspection] records shall include…the results.” If the test method is pass/fail, circling pass or fail makes sense, but if the test measures a dimension, the result should be a measurement. Also, if you have to investigate a complaint or non-conforming product, this dimensional information might be critical to the analysis.
The FDA has provided an official interpretation of these requirements in the QSR preamble:
“Comment # 147: One comment stated that record keeping this dimensional information might be critical to the analysis if you have to investigate a complaint or non-conforming product is a significant cost factor in the operation of a total quality system and that the revised CGMP regulation should not add cost through duplication of documentation. The comment said recording all quantitative data is inappropriate and of little value.
FDA agrees that unnecessary duplication of documentation should be avoided. They also believe that the Quality System Regulation requires the minimum documentation necessary to ensure that safe and effective devices are designed and produced. FDA similarly believes that maintaining records of results of acceptance activities is imperative to ensure that non-conforming product is not inadvertently used or distributed. FDA has, however, deleted from Sec. 820.80(a) the requirement for recording the results of inspections and testing, because Sec. 820.80(e) requires that the results of acceptance activities be recorded. The requirement in Sec. 820.80(a) was, therefore, unnecessary. Further, the regulation does not specify quantitative data, but simply requires that the results be recorded.
The FDA believes that it is essential for the manufacturer to maintain records which provide evidence that required acceptance activities were completed. These records must clearly show whether the product has passed or failed the acceptance activities according to the defined acceptance criteria. If a product fails to pass acceptance activities, you must identify the product as a non-conforming product and conduct an investigation. If the acceptance records are not clear about how the product failed, then the manufacturer may end up duplicating the acceptance to perform appropriate investigations.”
Here are three other methods that can save you time and add value.
Method 1: Run Charts
If you create an inspection results form that is in the form of a “Run Chart,” then you can put an “X” on the appropriate location of the Run Chart for each sample (see Chart 1 below). It is less time-consuming to write an “X” than the actual value. However, if you need to conduct an investigation, you can convert the “X” into a quantitative number and enter the values into a spreadsheet or statistical analysis software (e.g., Minitab). Also, inspectors and supervisors can visually glance at a Run Chart and determine if the measurement is “in control” or “out of control.” This is done by marking the upper and lower specifications on the Run Chart. Over time, alert limits can be established as a preventive action, as well. You can also use this data as a rationale for eliminating certain inspections, reducing sampling, qualifying suppliers, and even converting a part from statistical sampling to a “dock-to-stock” inspection.
One disadvantage of Method 1 is that it takes time to create inspection forms, and the forms need to be maintained as a controlled document, with the drawings for each part–as paper records or electronically. Therefore, I recommend that companies create a quality plan that calls for creating one of these charts every time an NCR is initiated for a part. That way, you only are creating this type of chart for parts that are found to be out of specification. This approach allows you to implement the work over a reasonable period of time. You can also habitually review historical data when you have an NCR that does not already have a Run Chart created.
Method 2: Automating Inspection Results
If you have critical inspection activities and a high volume of parts to inspect, you can automate recording measurements and performing data analysis. This can be done by purchasing digital inspection devices that automatically send the values to a computer system. Devices with this capability only require pressing a button to record the value, and the computer system will often provide the inspector with the sampling plan for each lot automatically. These sophisticated software systems require validation, giving manufacturers extensive real-time data on supplier performance, in-process inspection, and final product acceptance. The primary disadvantage of this method is the cost of installation and set-up.
Method 3: Pass/Fail with Go/No-Go Gauges
If a supplier can make good parts with high certainty, you may not need routine monitoring of part dimensions. In this case, you can reduce your inspection time by using a “go/no-go” gauge for critical attributes instead of measuring the dimensions. This type of gauge would be ideal if the tolerance for a part with a tolerance of +/- 3 mm. The length of a part can be verified to be between two lines, representing the upper and lower specifications for the tolerance. This method can also be used for precise tolerance if magnification is used. Still, performing a gauge R&R study of any go. If this type of inspection is used, you can use an inspection record that only records pass/fail. However, this inspection method is not recommended for parts that are occasionally out of conformity because re-measurement of parts will be necessary to investigate non-conforming products.
Statistical Techniques
The most significant advantage of methods one and two is that they facilitate statistical data analysis. Chart one shows too much variation for the tolerance of 6.50 mm to 6.60 mm. Some companies qualify suppliers for a new part by establishing a threshold for a minimum Cpk value (i.e., process capability coefficient). A typical Cpk minimum is 1.33. Often, the company will require that suppliers provide data for 100% inspection of the initial production lot. This data is then used to create a sampling plan based on the likelihood of parts being out-of-specification. High-risk dimensions might require 99.5% confidence, medium-risk dimensions might require 99% confidence, and lower-risk dimensions might require 95% confidence. Each confidence level corresponds to a different Cpk value. It is not possible to do this type of analysis for Method 3.
This article discusses the need to requalify EO sterilization validation and explains what is included in our EO sterilization procedure.
Your cart is empty
Do you need an EO sterilization procedure?
ISO 11135-1 is the international standard for sterilization validation for Ethylene Oxide (EO or EtO) sterilizers. The standard describes multiple methods of sterilization validation: 1) overkill approach, 2) single lot release, and 3) parametric release. The overkill approach is the most common method for validation of your EO sterilization process. The overkill approach is the method recommended by Medical Device Academy’s EO sterilization procedure. If you use a contract sterilizer, the sterilizer will already have completed an Installation Qualification (IQ) and an Operational Qualification (OQ). You must complete a Performance Qualification (PQ) for your product. A typical PQ for initial process validation consists of the following:
Process Challenge Device (PCD) validation
Bioburden measurement
EO residual measurement (as per ISO 10993-7)
Fractional Cycle (at least one)
3 Half Cycles
3 Full Cycles (or 1 Full Cycle, if performed in parallel with the three half-cycles)
Purchase the EO Sterilization Validation Procedure (SYS-031) – $299
SYS-031 EO Sterilization Validation Procedure
This procedure was updated in 2024 to include recent versions of various standards and to incorporate changes to make the procedure consistent with other procedures in Medical Device Academy's turnkey quality system. The updated procedure defines the requirements for ethylene oxide (EO) sterilization validation and revalidation/requalification outsourced to a contract sterilizer.
Price: $299.00
What do MPQ and PPQ mean?
In the ISO 11135 standard, steps #4 and #5 listed above are referred to as the microbial performance qualification (MPQ), while #6 is the physical performance qualification (PPQ). For a successful MPQ, some PCDs must be non-sterile after a fractional cycle to demonstrate the ability to recover the BI challenge organism. After a half-cycle, however, all biological indicators should be sterile.
What are BIs, CIs, and PCDs?
To avoid destructive testing, EO sterilization processes verify sterility by using process challenge devices (PCDs) located outside the device’s primary and secondary packaging. PCDs are more challenging to sterilize than the native bioburden on your device, and the PCDs can be quickly removed from a sterilized pallet without disturbing the wrapping of the pallet. PCDs are also referred to as external biological indicators (BIs). Biological indicators are used internally and externally to your primary sterile barrier packaging during the EO sterilization validation process, but only external BIs are used during routine EO sterilization. You can create your own customized PCD for devices that are especially difficult to EO sterilize (e.g., stopcocks), but you must verify that the PCDs are more challenging to kill than an internal BI in a fractional cycle. Commercially available PCDs often incorporate a chemical indicator (CI) into the label or the cap of the PCD, and some are incorporated into sophisticated tracking software with automatic incubators that read a barcode on the PCD label and detect the results of incubating the BIs in media. These systems are rapid, self-contained BIs that provide validated results in hours, minutes, or seconds.
Outsourcing EO sterilization and requalification
Ethylene oxide sterilization is usually outsourced to a contract sterilizer due to the environmental and safety requirements of working with EO. The contract sterilizer will provide a generic protocol for full validation that is compliant with ISO 11135-1. However, the ISO 11135-1 standard requires that manufacturers perform annual process reviews to evaluate the need to requalify/re-validate the sterilization process. Assuming there have been no problems or changes to the product or EO sterilization process, re-validation is not required at the end of the first year. However, companies are required to re-validate the process after two years–even if there have been no changes.
Longer frequencies for requalification cycles
If there have been no changes to the sterilization process, the product, or the biological indicators, then the manufacturer can use this as a justification for waiting until two years have elapsed before re-validating the ethylene oxide sterilization process. Also, there should be no evidence of sterilization failures or other problems with the validated process. However, that alone is not necessarily enough to justify extending the duration between validations beyond two years. Companies that justify intervals of three or more years have multiple products that use the same EO sterilization process.
In this case, the manufacturer may alternate annually between three, four, or even five different product families that are using the same EO sterilization process. In this case, one of the product families is being re-validated each year or every two years, but the interval between validations for any one product family is longer. This approach is valid if the products are made of similar materials and use the same EO sterilization process. If you only have one product, then you need to re-validate the sterilization process once every two years to verify the process remains active.
Minimum revalidation requirements
When you determine that it is time to re-validate your ethylene oxide sterilization process, you need to perform the following tests to meet the minimum requirements of ISO 11135-1:
Re-validation of PCD
Bioburden measurement
EO residual measurement
1 Half Cycle
1 Full cycle (to verify the EO residuals are acceptable)
The purpose of #1 is to verify that the resistance of internal BIs used in the half-cycle is more resistant than the product bioburden. The purpose of #2 is to verify that bioburden levels have not changed, and the type of organisms has not changed. In practice, most companies monitor bioburden quarterly, and therefore this step should be routine. Step 3, EO residual measurement, should be performed to verify that there have not been minor changes to the product or process that would increase the concentration of EO, Ethylene Chlorohydrin (ECH), or Ethylene Glycol (EG) beyond the Tolerable Contact Limit (TCL). The purpose of this third test is to prevent localized irritation caused by residual chemicals from the ethylene oxide sterilization process.
Step 4 of the re-validation is intended to verify that a full injection of EO is more than required to kill the bioburden present for the number of injections required for a half-cycle.
The final step is to perform a full cycle. The product from the full cycle is typically used for EO residual testing. Any product from the full cycle that is not used for testing can be sold after sterility testing is complete.
Partial loads & rework
If you occasionally sterilize loads that are less than “full loads,” then you need to ensure that you have validated a minimum load or a specific partial load (e.g., half-pallet, instead of a full pallet). In the case of a partial or minimum load, you may identify different locations in your load that is considered “worst-case.” These are the locations that had PCDs that were not sterile in a fractional cycle.
Most companies do not have concerns about the cost of the actual sterilization runs during re-validation, and biological indicators are typically less expensive than boxes of products. The primary cost concern for re-validation is any product that must be scrapped. Therefore, many companies will accumulate dunnage (i.e., empty packaging or scrap product) over time to fill a sterilizer. This dunnage may be used to ensure that every load is full, or it may only be used for re-validation.
Another alternative to using dunnage for re-validation is to validate a rework process. Any product exposed to a fractional or half-cycle can be re-sterilized in a full cycle. To justify the commercial use of that product, a company needs to validate that the product will not be damaged by exposure to two full cycles. One of the key acceptance criteria for rework is the EO residual levels in the product. However, the manufacturer also needs to determine if any product deterioration by a second exposure to EO would affect performance.
Other EO sterilization considerations
Many companies do a poor job of reviewing the potential impact of changes to a product, packaging, and biological indicators. Ideally, initial validation involves different lots of product, packaging, and biological indicators to assess lot-to-lot variability. However, the packaging and biological indicators often consist of only one lot during validation. Minor changes to the tolerances may reduce the amount of ethylene oxide that is absorbed by the product or change the resistance of the biological indicator to the sterilization process. Therefore, these minor changes should trigger a re-validation.
Changes in suppliers with the same specification can also be difficult to evaluate. If a component is made of a material that absorbs EO, then it may be recommended to re-validate sterilization for any changes to suppliers of those components. Re-validation in these cases may consist of only a fractional cycle, half cycle, or full cycle to evaluate risks associated with the change.
Who should evaluate the need for EO sterilization requalification?
Evaluating the need for re-validation should include inputting three types: 1) microbiological, 2) materials, and 3) performance. To make these assessments, typically, a cross-functional team is needed. Someone responsible for design and development can assess the performance impact of changes. A materials engineer is generally needed to assess the interaction between components and EO. Finally, a microbiologist is needed to confirm that there is no impact related to biological indicators or bioburden.
Additional Sterilization Training
Medical Device Academy has two webinar recordings related to sterilization validation:
If you are a third-party or hospital reprocessor, learn how to prepare an FDA eSTAR 510(k) submission for reprocessed single-use devices.
Why is there so much interest in reprocessed single-use medical devices?
With increasing pressures on the medical device industry to make healthcare more affordable, there has been a push to reprocess and reuse single-use devices. Reprocessors obtain used devices from healthcare facilities. The reprocessors clean, process, resterilize, repackage, and relabel devices. Reprocessors must obtain FDA 510(k) clearance by demonstrating that the safety and effectiveness of the reprocessed device are substantially equivalent to the single-use device produced by the original equipment manufacturer (OEM). The FDA created a FAQ document for single-use devices, and three guidance documents were published:
Why do reprocessors have difficulty preparing an FDA eSTAR for reprocessed single-use devices?
Obtaining 510(k) clearance for a device your company did not design can be challenging because the reprocessor doesn’t have access to all of the required design and manufacturing information. The following sections of the FDA eSTAR submission pose unique challenges for reprocessed single-use devices:
Labeling – What should and should not be included in the reprocessed device labeling
Biocompatibility – How to identify the materials, and determine what biocompatibility testing needs to be done
Performance Testing – Strategies for determining appropriate performance testing
Labeling Section of the FDA eSTAR for reprocessed devices
Labeling of reprocessed devices consists of the instructions for use and the packaging label(s). Device package labeling may also direct the user to both the reprocessor’s IFU and the OEM’s IFU. If you are referencing the OEM’s IFU, it is also important to include the OEM’s model number. Instructions for use should consist of:
Indications for use, which must be equivalent to the OEM indications.
All of the necessary warnings and cautions and basic operating instructions needed to operate the device safely.
The instructions for use may also instruct the user to reference the OEM instructions for use for additional information.
Instructions on the handling of the device after use, with the likelihood that the device will be returned to the reprocessor to repeat the cycle.
Biocompatibility Section of the FDA eSTAR
Biocompatibility data is more challenging to provide if you replace or modify original components. If reprocessing does not modify the OEM device whatsoever, you can claim that the materials are identical to the OEM device. Therefore, the reprocessed device does not require biocompatibility testing. However, the reprocessor still needs to evaluate the biological risks associated with the reprocessing of the device by testing for cleaning and sterilization residuals. This involves testing for cleaning agent residuals and EO residual testing (ISO 10993-7), if applicable. If applicable, this involves testing for cleaning agent residuals and EO residual testing (ISO 10993-7)
If you replace any of the components during reprocessing with a new component that is identical in dimension and material to the OEM component, minimal biocompatibility testing will be required. If the exact material used by the OEM is unknown, reprocessors can perform material identification testing to determine the material used, and then create the replacement part out of the same material.
If you modify or replace any patient-contacting components on the device such as lubricants, insulation, etc., with components that are different from the OEM, then you will need to perform additional biocompatibility testing to prove that the new or modified material is biocompatible. This testing will depend on the duration of contact and where will the material contact the patient. The new material will also need to be listed in your device description and Section 15 of your 510(k) submission.
Performance Testing Section of the FDA eSTAR
There are three primary sources for identifying performance testing requirements of reprocessed devices:
OEM Testing listed in the OEM 510(k) submission
Predicate Testing listed by another reprocessor of an equivalent device
Product Standards listed under the product classification code for the reprocessed device or the OEM device
You should reference a predicate device that has been reprocessed and the OEM device to identify performance testing. Some testing is specific to the functional performance of the device. For these tests, you need to compare performance side-by-side against the OEM. Another testing is specific to reprocessing; you will reference the predicate device. Sources of information regarding the required tests for each of these devices can be found in the 510(k) summaries of the respective devices. If possible, it’s helpful to select a predicate that has a redacted 510(k) available on the FDA’s website. If a redacted 510(K) is not readily available, you may request a redacted copy through the Freedom of Information Act online. A redacted copy of the OEM 510(k) is also helpful. It’s helpful to select a predicate with a redacted 510(k) available on the FDA’s website if possible
If testing information is not as readily available in the 510(k) summary, you will determine the essential performance functions of the device, and design tests to evaluate and compare the OEM device and the reprocessed device for those functionalities. Some devices have specific standards for their design and/or testing. To determine if the reprocessed device has any applicable standards, you should search the product code of the reprocessed device and the product code of the OEM device, if they are different, in the FDA product classification database. The search results will list recognized standards applicable to the reprocessed device.
Additional tests that may be needed to validate reprocessing include residual protein, residual carbohydrates, and the presence of hemoglobin. These tests ensure that all biological material from previous use is removed. If you are not performing biocompatibility testing on the reprocessed device, you must do a chemical test to ensure no residual detergent or cleaning residues remain on the device. You must also determine how many reprocessing cycles the device can survive before performance degradation. This can be done by repeating simulated use, reprocessing, and performance testing until a statistically relevant decrease in the performance of the device is observed.
If you have additional questions regarding preparing your 510(k) submission, please visit our Contact Us webpage to schedule a meeting with Lindsey Walker or Rob Packard.
In this article, you will learn how to spend your time during a supplier audit. We’ll teach you what is important and what you can skip.
Which suppliers need a supplier audit?
Before you start scheduling supplier audits, you should first decide which suppliers you need to audit. You are required to use a risk-based approach for supplier quality management but have specific recommendations. We recommend that you create five risk-based supplier quality categories:
Critical suppliers
Crucial suppliers
Off-the-shelf component suppliers
Service providers
Consultants
Your critical suppliers are contract sterilizers, contract manufacturers, and contract packagers. Your crucial suppliers are suppliers manufacturing custom components or subassemblies. Off-the-shelf components speak for themselves, but examples of service providers include a company doing plating and other secondary processes. The last supplier category, but not the least, is the consultant category, such as the quality system auditors you hired to do an internal audit.
Which supplier categories require a supplier audit?
The FDA regulations don’t specifically require supplier audits. However, if an inspector finds any nonconformities among your purchased components, you will need to demonstrate how you have addressed the quality issues. If the corrective actions taken are not sufficient, you will need to conduct supplier audits as part of your corrective action plan or effectiveness check. Other countries have different expectations with regard to supplier auditing, but the most common supplier categories that you will be conducting a supplier audit of are “critical suppliers” and “crucial suppliers.” These two supplier categories are also the two supplier categories that you will need to make sure are prepared and willing to accommodate unannounced audits by Notified Bodies. Click on the image below if you would like to read the requirements for audits conducted by Notified Bodies.
What is the purpose of a supplier audit?
When you attend a lead auditor course, the focus is on quality system auditing. However, when you perform a supplier audit—the quality system is not the focus. The focus of a supplier audit can fall into two primary categories: 1) qualifying the supplier or 2) re-evaluating the supplier.
Suppliers are not required to have a registered quality system or ISO 13485 certification. Therefore, many of the things that an auditor might learn about audit agendas in a lead auditor course just don’t apply. However, one thing always applies: reviewing previous quality issues. When we audit internal auditing and supplier auditing programs, we find that one of the most common mistakes is the failure to close out previous nonconformities. Therefore, the second section of my audit report template is a review of prior audit findings. If you have no previous findings, ensure your audit report states that. If you are qualifying a new supplier, ensure that the new supplier doesn’t have the same problems you are having with current suppliers.
When you close the previous issues, there are two approaches. The first approach is to close previous issues at the beginning of the audit—immediately after the opening meeting. This is the most common strategy. The second approach is to close previous issues as you audit the applicable area. For example, if you have previous problems in the area of incoming inspection and maintenance records, it might make sense to close these findings when you audit these areas. The advantage of this second approach is that it ensures that the process owner is closing the previous finding and facilitates the sampling of additional records.
What has little value in the supplier audit agenda? Auditing the management review process has the least value because the supplier is not required to have a quality management system. In fact, subcontractor audits for BSI do not include management reviews, CAPAs, or internal audits—the three required areas for every quality system audit.
What are the most valuable areas to audit?
Incoming inspection, control of nonconforming materials, preservation of the product, production controls, training, and process validation are the areas we typically audit. We would like to start with the nonconforming material area and see which materials are on hold. Then, we would like to sample the incoming inspection records for those raw materials. Next, we want to see how the company is storing those raw materials—if they are accepted. We typically cover these three areas as one process approach audit. This also happens to be the process audit we like to use for training new auditors because the audit of incoming inspection results in numerous audit trails in the support process areas of document control, training, calibration, etc.
The next area we visit is the production area. For this portion of the audit, we are doing a process audit of the production process. We usually request that we schedule the audit for a time when the production area is running the product(s) of interest. A process flow chart helps plan this portion of the audit, and we will often write some notes directly on a copy of the process flow chart.
We conclude the audit with follow-up trails in the areas of 1) document control (to ensure the supplier has the most current versions of all documentation “we” provided), 2) calibration (to ensure that all measurement devices used for inspection are calibrated), and 3) training (to ensure that all personnel working on “our” product are appropriately trained).
What are the advantages and disadvantages of skipping areas?
Since we do not have to spend time on quality system issues during a supplier audit, we spend more time sampling records in the other areas. Therefore, we might sample 5-10 records in each of the above areas instead of 3-4 records. If the number of samples available to sample is small, we may even sample 100% of the records. We also have a supplier auditor tool kit to help your supplier auditor team prepare.
Did you consider confidentiality and security issues during your supplier audit?
Historically, it has always been easy to identify a missing or out-of-date confidentiality agreement during audits, but do you include this in your internal and supplier audits? The new cybersecurity requirements that the FDA released in October 2023 certainly changed what companies need to provide in a 510(k) submission, and the latest FDA eSTAR template has a lot of specific documentation that companies need to include their 510(k). If you want to learn more about the 510(k) requirements, please visit our webpage for the cybersecurity work instruction and webinar.
How will this impact your supplier audit program?
Do you include cybersecurity questions in your supplier audits?
Do your supplier quality agreements address cybersecurity?
Do you have cybersecurity testing vendors added to your approved supplier list?
Is cybersecurity embedded in your post-market surveillance activities?
Do you and your supplier have a schedule for cybersecurity retesting?
This procedure case study describes an error-proof method for procedure review and approval of quality system procedures.
My first training in procedure review
The first time I was formally trained on how to conduct a procedure review was during a lead auditor course. I thought the topic of procedure review seemed out of place, but as I audited more companies, I realized that missing regulatory requirements in a procedure are quite common. Regardless of who reviews a procedure, or how many times it is reviewed, something is always missed. Unfortunately, a desktop audit of procedures is not an effective corrective action or verification method. Auditing procedures is an ineffective method for reviewing procedures because audits are limited by sampling.
A better approach to procedure review than auditing
Instead of random sampling, a systematic review of 100% of regulatory requirements is needed to ensure that none of the regulatory requirements are accidentally omitted. Systematically reviewing regulatory requirements for each country your company is selling in is tedious at best. You need a tool to make the reviewing process error-proof and straightforward. You also need each procedure reviewer to have a defined function to eliminate the duplication of work.
Procedure reviewer and approver roles
There are 3-5 reviewers of procedures in most companies. Some companies make the mistake of having as many as 8-10 reviewers of procedures, but more is not better. There are four primary roles for procedure review, but you could have as few as two people approving most procedures:
process owner (must review and approve)
quality management (must review and approve)
regulatory (must review, but optional approver)
independent (optional review, but not an approver)
You are not required to have all four of these reviewer roles, but including these four roles in your document control process is a best practice. Differentiating between reviewers and approvers should also be considered in your document control procedure. The only documents we recommend top management be a reviewer and approver of are:
Quality Policy
Risk Management Policy
Quality Manual
Management Review Procedure
The reason for top management reviewing these four documents is because top management has a regulatory responsibility related to each of these documents.
Process owner role
The process owner is the owner of the procedure for that process. Therefore, the process owner needs to approve that procedure. It would make no sense to own a process without the ability to approve changes. The process owner may also be the procedure author, but we don’t recommend it. Editing someone else’s work is more effective than editing your own work. Instead, we recommend that the process owner delegate the responsibility for writing and updating procedures to a subordinate who performs the procedure. Then, the process owner is responsible for reviewing and approving the procedure.
Quality management role
The quality management person needs responsibility for reviewing and approving all procedures because this person is responsible for the entire quality system. They need to make sure the procedure is accurate in the context of the entire quality system. The quality management person is the best person to review interactions with other processes. For example, the management review process has twelve required inputs (i.e., ISO 13485, Clause 5.6.2A-L). Each of those inputs comes from another process and procedure. It is essential to ensure that if you are reviewing the complaint handling procedure, somewhere in that procedure, it should state that the monitoring and measuring of complaint trends should be input into the management review process.
Regulatory role
Usually, the regulatory person is responsible for verifying that a procedure meets 100% of the regulatory requirements. This person should verify that the scope of the procedure identifies the relevant markets. If there are references to documents of external origin, the regulatory person should verify that these references are accurate. The best way to do this is by performing a gap analysis. Sometimes the quality management role and the regulatory role are combined in a small company, but larger companies will keep these roles separate. Just because the regulatory person performs a gap analysis as a reviewer, that doesn’t automatically translate to the need for approval of the procedure. We recommend making the decision on whether a regulatory person should approve a procedure based on whether the procedure has specific regulatory requirements (e.g., annual registration or regulatory reporting).
Independent reviewer role
Finally, the independent reviewer is looking for two things:
Does the procedure make sense–to someone who performs the procedure (if that person was not the author); and to an external auditor, such as a certification body (internal auditors can fill this role)?
Are there typos, spelling, or grammar mistakes?
The independent reviewer does not need to be a manager. It needs to be someone who writes well. Editing is tedious, but apparent mistakes in spelling or grammar prompt auditors to review procedures more carefully. If available, we recommend asking an internal auditor to be the independent reviewer. Depending upon the experience of the independent reviewer with regard to performing a gap analysis, the person with regulatory responsibility may delegate the task of gap analysis to independent reviewers. This role can also be satisfied by a consultant with technical writing ability. Medical Device Academy’s resident expert at this is Matthew Walker.
Procedure case study – The most common auditor findings
The two most common reasons for audit findings are:
the procedure is not being followed, and
a regulatory requirement is missing from your procedure.
Not following the procedure
The first problem is the most common reason for audit nonconformity, as companies include requirements in the procedure that are not regulatory requirements. Auditors look for objective requirements to audit. Therefore, if you include objective requirements in your procedure an auditor is more likely to select those requirements to sample than subjective requirements–even if the requirement is not a regulatory requirement. This is one of the reasons we recommend having processing owners review and edit procedures. If you purchase a procedure, it’s important for the person who will be performing the procedure to carefully review the procedure to ensure it matches how they intend to perform that process. If it’s a manufacturing procedure, we recommend training personnel with a draft procedure and handing out red pens. That also dramatically reduces complaints from the people who do the work.
Regulatory requirements missing
For regulatory requirements, your regulatory reviewer needs to create a checklist that includes 100% of the requirements for that procedure. This approach is called a gap analysis. The model for gap analysis documentation we like to follow is the General Safety and Performance Requirement (GSPR) Checklist used for technical documentation (i.e., for CE Marking). There are 23 GSPRs in the MDR and 20 GSPRs in the IVDR. Most of the GSPR requirements have multiple subparts. The regulatory person who completes the GSPR Checklist must indicate the following information next to the applicable requirement in the checklist table:
yes, the requirement applicable or justification if it’s not applicable
a reference to any applicable standards
a cross-reference to the record where evidence of meeting the requirement can be found (e.g., the risk management file)
Regulatory personnel can revise this approach slightly by doing the following for the review of procedures:
yes, the requirement applicable or justification if it’s not applicable
a reference to the applicable specific sub-clause in a Standard or a regulation
a cross-reference to the subsection of the procedure where evidence of meeting the requirement can be found (e.g., section 5.1 of the SYS-003)
Procedure Case Study of the Management Review Procedure (SYS-003)
In Medical Device Academy’s Management Review Procedure, Section 8 is the “procedure section.” Sub-section 8.3 of the procedure lists all the required inputs for a Management Review meeting. Next to each input, we included a cross-reference to the sub-clause in ISO 13485:2016 for the Management Review input.
There is also a requirement in ISO 13485:2016 for conducting Management Reviews at scheduled intervals. This requirement is met by sub-section 8.1 of the Management Review procedure. We used the same approach to identify and cross-reference to this requirement.
Teaching auditors by performing your own procedure case study
Now, when we teach our Lead Auditor Course, we ask attendees to split into small groups to review a procedure–one procedure for each group. In one of the companies where we did this, each of the four teams found a regulatory requirement that was missing from the procedures they were reviewing. All four procedures the teams selected were already reviewed, approved, and currently in use at the time of the auditor training. The four teams created their own procedure case study to demonstrate the importance of reviewing procedures for regulatory requirements.
The author discusses his personal experience with quality and regulatory training and shares his secrets for better instructor engagement.
What is instructor engagement?
Instructor engagement is a term that describes actions taken by the instructor to involve their class, whether the training is on-line or in person. Instructor engagement includes six basic elements:
Training content and format
Verbal communication
Non-verbal communication
Training environment
Audience
Audience involvement
Any one of the above six elements can ruin a training class, but a great instructor can compensate for weaknesses in any one area by taking advantage of the other elements.
To be successful, you need to hook your audience in the first ten seconds
It is crucial to engage your audience in the first ten seconds. When the audience is live, if you don’t engage them immediately they will find something on their phone to distract them. If the audience is on-line, they will swipe to the next video in their feed. Toastmasters suggests beginning your presentation by using one of five methods for hooking your audience:
Tell a story
Make a bold statement
Ask a question
Get the audience to laugh
Ask the audience to visualize something
Consequences of poor engagement
If an instructor does not engage students, the best case scenario is that the training will not be effective. In other words, the class will not learn the material being taught or they will retain the information for less than 24 hours. For the student, their time and money was wasted. For the instructor, they will feel exhausted at the end of the training and they will have trouble finding future training jobs.
Results of good instructor engagement
If a class is engaged in training they will learn the material, retain the information in their long-term memory, and the will recommend the instructor to other people that are interested in learning about the topic. For the student, their time and money was well spent. For the instructor, they will feel energized at the end of the training and students will come up to them at the end of the training asking for a business card and to discuss future training opportunities.
Where can you find an examples of good and bad instructor engagement?
When I first wrote a blog on this topic it was 2012 and there were very few blogs and almost no videos dedicated to quality systems or regulatory affairs. Twelve years later, almost nobody reads blogs and there are multiple competitors that publish new videos weekly. The primary channel for watching videos is YouTube, but YouTubers are simultaneously live-streaming on YouTube, LinkedIn, and Facebook. These channels are the best place to find examples of good and bad instructor engagement. You will probably have a strong opinion about the quality of the speaker in the first ten seconds, but the algorithms that guide your surfing of these platforms will automatically steer your viewing to the best videos. These videos are not considered to be the best solely on content and format. The amount of audience engagement is the biggest driver. The algorithm recommends videos based upon the percentage of audience retention, the number of comments by viewers, and the number of people that share the link to your video.
Why doesn’t Dr. Shulman’s video rank higher in the algorithm?
The video I embedded in this post has only 554 views currently, but it has been posted on YouTube almost six years. Why doesn’t this video rank higher and get recommended by YouTube to more people? Because there is no involvement of the original audience or the current YouTube audience. Despite the talented speaker and the use of video with clear PPT slides, there are no comments on YouTube. The content and format is good, and the verbal communication is good. However, RAPS has Dr. Shulman standing behind a podium at the event so the non-verbal communication is not as strong as it could be. In addition, large conferences are typically one of the weakest environments for encouraging instructor engagement with the audience. The video could receive much higher rankings if the original audience was involved in the video as well. If the question and answer session were included, that might have helped. RAPS could also improve the video’s performance by adding more details to the description of the video. Finally, the performance of the video is impacted by the number of subscribers to the RAPS YouTube channel and the engagement of those subscribers. To demonstrate this, I will add a comment to the video. That should cause a small increase in engagement and viewers.
How can you improve your internal quality system training?
Anyone can read and understand a procedure, but this is the least effective method of training people. You could also have employees watch a training video, but quality assurance and regulatory affairs are among the most boring topics on planet earth. Most of the training out there is “Blah, blah, blah…” and “Death by PowerPoint.” Instructor engagement for that type of training is poor, and it could get you fired. Don’t read your slides, don’t turn your back on the audience (or they’ll attack), and PLEASE don’t ever ask someone to read the definition of nonconformity out loud to the rest of the group. Inspire and engage the class. You need to get your audience to pay attention, ask questions, and share their own thoughts out loud. For example, instead of using a PowerPoint, try displaying the actual procedure and ask an audience member to find each of the points you are teaching in the procedure. You could even teach them a cool search tool (i.e., CTRL + F) to find the content. You might even adding a symbol to the procedure to help them find those requirements.
Nine ways to improve your own instructor engagement
If you are hiring a consultant to help you with quality and regulatory training, then you certainly want to hire an expert. However, it is more important that the speaker is engaging. Knowing this fact, you could try improving your own presentation skills to achieve higher instructor engagement for a lot less money. I’m six-foot, six inches tall, and I have a loud booming voice. My mother has red hair, and she was an opera singer. I’ve got the voice to fill any auditorium and stage presence to match. But you don’t have to be big, tall, or loud to capture the attention of your audience. Here are my top nine ways to improve instructor engagement:
Practice vocal variety
Move, don’t stay stationary
Ask the audience questions
Use anecdotes, case studies, and stories
Try using props
Take breaks
Plan a surprise
Force feed the audience legal stimulants
Give students homework
What is vocal variety, how does it impact instructor engagement?
Vocal variety is more than just the volume of the speaker. Vocal variety consists of pitch, tone, volume, and pace. Generally I speak too fast and my voice is very loud. Therefore, if I want to emphasize a point I can exercise two changes in my voice to immediately capture the attention of students: 1) speak softly, and 2) speak slowly. Another approach I have used, is to speak slowly and repeat myself, but the most dynamic way to get your audience’s attention is to stop speaking for a few seconds. Silence is powerful.
Many people struggle to understand how to vary their pitch, but they’re overthinking it. When we ask a question, we raise the pitch of our voice slightly at the end. This upward inflexion of our voice signals to listeners that we are asking a question. For example, if you repeat the last one to three words in the other person’s sentence, and you say this with an upward inflection, the listener will perceive that you are asking a question to better understand what they mean. This is much more effective than asking “what do you mean?” For example, if a student in the class says “Is it enough if we perform annual reviews?” As the instructor you can employ the technique of mirroring by asking, “Annual reviews?” This will encourage the other person to elaborate on what they meant by “annual reviews.” This technique ensures that you know what the student meant, and it gives you time to consider what they said before you respond. This is also a strategy recommended by expert negotiators.
Movement is attractive
Most of our brain power is dedicated to processing what we see–not what we hear. Therefore, listeners are more likely to notice when you move. You can jolt your audience awake simply by stepping out from behind a podium or changing your movement pattern (e.g., occasionally moving forward instead of pacing side to side). Movement also includes body language. You can modify your posture, stance, and position in front of the audience to communicate information non-verbally. You can use gestures to communicate non-verbally or you can use facial expressions. For example, if you frown and shrug your shoulders, what does that mean?
Questions are essential for instructor engagement
No matter how smooth and eloquent your voice is, nobody wants to hear only you speak. This is one of the reasons conferences have multiple speakers. However, during a single presentation you can get students to participate and present some of the information by asking them questions. There are a few techniques that help get the audience to speak more. Instead of asking a question to the group as a whole, try pointing to a specific person, ask them their name, and then ask them a question. Second, in order to “break the ice” at the end of your presentation, have some “seed questions” prepared. If you are conducting a webinar, seed questions can be read by you. In a live presentation, you can give a list of seed questions to your host or friends in the audience. Finally, you can also begin your training with a question (i.e., Toastmaster hook #3 above).
Tell a story and make a point
For each teaching point you should consider using an anecdote, case study, or a related story. The audience will want to know how the story ends, and they are more likely to remember the story. If you haven’t conducted more than 1,000 audits, traveled all over the world, or have more than 25 years of experience–don’t despair. You can always interview other people on any topic to get their stories. If you are looking for tips on how to construct a story, Toastmasters comes to my rescue again. Here’s their recommended six-part structure for the Hero’s Journey:
Setup
Inciting incident
Progressive complications
Insight
Climax and resolution
Lesson
What kind of prop can you use for quality and regulatory?
In one of the paragraphs below, I mention a simple prop that you can use for training–product samples. I did this in one of our live-streaming YouTube videos where I explained how to review medical device labeling. But you can use other things in your environment. For example, the first public speaking course I ever had was taught by a man with Polio that used crutches. He used one of his crutches as a prop to demonstrate how he looked over a fence.
How often should you take breaks?
Students cannot maintain a state of alertness and attention indefinitely. Your body naturally cycles between higher and lower alertness every 90 minutes. After 90 minutes it becomes harder to focus and you need to take a break. If you can, splitting 90 minutes into two 45-minute sessions is even better. You can also experiment with two strategies for better instructor engagement: 1) conduct a “pop quiz” after a break to make sure the audience understood the information they were just taught, and 2) don’t be afraid to adjust the breaks slightly to coincide with a change in topic. Changing topics at a break allows you to repeat the most important points three times. The first time when you introduced a topic, the second time when you have concluding remarks at the end of a topic, and a third time after the break when you make sure the audience understood the material you presented before the break.
What kind of surprise will engage your audience?
There are two strategies for using a surprise: 1) promise to surprise them in the future, or 2) don’t tell anyone until you surprise them. The first strategy works best when you are trying to get people to watch until the end of the training. This is commonly used by YouTubers to get people to watch the video until the end. Unfortunately, this backfires because we can fast-forward to the end. I’m not suggesting that you shouldn’t surprise your audience, but the surprise needs to delight your audience. You might also need to surprise and delight your audience more than once. Even then, some of your audience will still nod off and completely ignore you. When this happens, throw a Snickers bar at the offending student.
This is an essential tool for any instructor. It functions as a tool to prod sleeping students awake, is small enough to cause minimal injury when thrown, serves as an emergency food supply, and is gluten-free.
How to force-feed students legal stimulants
If legal counsel recommends against using projectiles to encourage class participation, you might also consider one of my all-time genius ideas–consuming dangerously large quantities of caffeine. I was scheduled for a two-day course in Ottawa, but the day before I needed to perform an audit in Pennsylvania. My flight was the last flight into Ottawa, which arrived at approximately 1 o’clock in the morning. My arrival was delayed an additional hour in customs by the person in front of me who was trying to smuggle an extra carton of smokes into the country. Just before 4 a.m., my taxi arrived at the Albert at Bay Suite Hotel. The class started at eight in the morning. I made it to class on time, and the excessive consumption of several pots of black coffee helped get me to lunch. Then my legs started getting a little shaky. Fortunately, there was a convenience store next door that sold my favorite chocolate–the Dark Aero bar! After four of these monstrous doses of cacao, and another pot of coffee, I could have listened to the lecture on the Canadian Medical Devices Regulations all night. The only problem is that my hands are still shaking 15 years later.
Hershey’s copied them, but the result was a mere shadow of Nestle’s greatness. Canadians know how to make junk food, tell a joke, and play hockey!
Why was the instructor engagement high in Ottawa?
Despite the physical handicap of sleep deprivation, I still learned a ton from my course in Canada. Here’s why:
The instructors were both regulatory experts that were able to share anecdotes, case studies, and stories about real-world application of the Canadian Medical Devices Regulations. One of the instructors even worked for Health Canada.
The audience was hyper-motivated to pass the course, because everyone in the class worked for a Notified Body that had sponsored them to take the course. In order to stay employed and get a raise, I needed to pass that course. If I failed the exam, I had to absorb the cost to travel back to Ottawa and retake the course in February (BRRRR!).
The instructors brought more than a dozen medical devices to the class. These props gave us something to read, touch, and ask questions about. The instructors broke us up into small teams to study the labeling and instructions for use of each device. Even students from Japan, Europe, and Australia were familiar with some of the products. This was critical because we all needed to be able to identify incorrect Canadian labeling.
The best instructor engagement tool used was humor. The instructor from Health Canada was hilarious. He had everyone laughing at his jokes for the entire course. Most of the jokes were not funny enough for a stand-up routine, but this was a mandatory regulatory course on Canadian regulations. Who would even expect a chuckle? Despite the strengths of these instructors, there is only one reason why I know the Canadian Medical Devices Regulations (CMDR), as well as I do. I use them every single week.
When students are forced to do homework, they will pay attention
After completing my CMDR training, I had to audit 162 days for BSI in 2011. Ninety percent of those 162 days were for companies that required a Canadian Medical Device License. This forced me to use the information I learned in the course. I was also consulting for companies at the same time I was auditing for BSI. Consulting clients hired me to prepare and submit the Canadian Medical Device License Applications for them. I also had to create procedures for Canadian Licensing, Incident Reporting, and Recalls. I spent another 60+ days in 2011 doing consulting, which helped me hone my knowledge of Canadian device regulations.
Teaching others will make you a guru
Most people are terrified to speak on any topic–even to a small group of coworkers. However, I believe that teaching others is the secret to becoming a guru on any topic. I was one of BSI’s instructors that taught the regulatory comparison course from 2010 to 2012, which compared the regulations of the USA, Canada, Europe, Australia, and Japan. Therefore, at least once a month, I had a classroom of 6-20 people asking me challenging questions about how to interpret and apply regulations from each of these countries to their products. I used every bit of knowledge I learned in that course in Ottawa, and I started using that knowledge immediately after the course. Peers, clients, and students challenge my knowledge of these topics every day. This is what makes you a subject matter expert. If you need to learn something about quality assurance or regulatory affairs watching a one-hour webinar, reading a blog, taking a five-day course, or shadowing another more experienced person is not enough. In the end, all of the above will get you to the level of barely competent! If you want to master any topic, you need to practice instructor engagement and use everything you learn for several years.
Data hazards occur when there is missing data, incorrect data, or a delay in data delivery to a database or user interface. This can occur with both software as a medical device (SaMD) and software in a medical device (SiMD). Missing data, incorrect data, and delays in data delivery cannot cause physical harm to a patient or user. Therefore, many medical device and IVD manufacturers state in their risk management file that their device or IVD has low risk or no risk. This is an incorrect software risk analysis because failure to meet software data or database requirements results in a hazardous situation, such as 1) an incorrect diagnosis or treatment, or 2) a delay in diagnosis or treatment. These hazardous situations can compromise the standard of care at best, or at worst, hazardous situations can result in physical harm–including death.
Where do you document these hazards?
Data hazards are documented in your software risk management file, but the data hazards are referenced in multiple documents. Usually data hazards will be referenced in a software hazard identification, the software risk analysis, software verification and validation test cases, and the software risk management report. Security risk assessments will also identify potential data hazards resulting from cybersecurity vulnerabilities that could be exploited.
How do you identify data hazards?
IEC 62304 is completely useless for the purpose of identifying data hazards. In Clause 5.2.2 of the standard for the software life-cycle process, the only examples of data definition and database requirements provided are form, fit, and function. IEC/TIR 80002-1, Guidance on the application of ISO 14971 to medical device software, is extremely useful. Specifically, in Table B1 the following potential data hazards are identified:
Mix-up of Data such as:
Associating data with the wrong patient
Associating the wrong device/instrument with a patient
Associating measurements with the wrong analyte
Loss of data resulting from events such as:
Connectivity failure or Quality of Service (QoS) issues
Incorrect data acquisition timing or sampling rates (i.e., measurement)
Capacity limitations during peak loads
Missing data fields (i.e., incorrect database configuration or database mapping)
Modification of data caused by:
Data entry errors during manual entry
Automated preventive maintenance
Reset of data
Rounding of data
Averaging of data
Patient data also presents a security risk. Access to data must be controlled for data entry, viewing, and editing. Other potential causes of data integrity issues include power loss, division by zero, overflow/underflow, floating point rounding, improper range/bounds checking, off-by-one, hardware failure, timing, and watch-dog time outs. In Table B2 the guidance provides additional examples of causes and risk control measures are recommended for each cause.
Data hazards associated with artificial intelligence (AI) and machine learning (ML)
There are also data hazards associated with AI/ML software. When an algorithm is developed there is a potential for improving the algorithm or making the algorithm worse. There is always a data bias resulting from the patient population selected for data collection and the clinical users that assign a ground truth for that data. Sometimes the data entered is subjective or qualitative data rather than objective, quantitative data. The sequence or timing of data collection can also impact the validity of data used for training an AI algorithm. AAMI CR 34971:2023 is a Guide on the Application of ISO 14971 to Machine Learning and Artificial Intelligence. That guidance identifies additional hazards associated with data and databases.
How are these hazards addressed in software requirements?
In IEC 62304, Clause 5.2.2, lists the content for twelve different software requirements (i.e., items A-L):
a) functional and capability requirements;
b) SOFTWARE SYSTEM inputs and outputs;
c) interfaces between the SOFTWARE SYSTEM and other SYSTEMS;
d) software-driven alarms, warnings, and operator messages;
e) SECURITY requirements;
f) user interface requirements implemented by software;
g) data definition and database requirements;
h) installation and acceptance requirements of the delivered MEDICAL DEVICE SOFTWARE at the operation and maintenance site or sites;
i) requirements related to methods of operation and maintenance;
j) requirements related to IT-network aspects;
k) user maintenance requirements; and
l) regulatory requirements.
These software requirements may overlap, because any specific cause of failure can result in multiple types of software hazards. For example, a loss of connectivity can result in mix-up of data, incomplete data, or modification of the data. Therefore, to ensure your software design is safe, you must carefully analyze software risks and evaluate as many test cases as you can to verify effectiveness of the software risk controls.
What is the best way to analyze data risks?
There are multiple risk analysis tools available to device and IVD manufacturers (e.g., preliminary hazard analysis, failure modes and effects analysis, and fault-tree analysis). Using a design failure modes and effects analysis (i.e., dFMEA) is the most common risk analysis tool, but a dFMEA is not the best tool for software risk analysis. There are two reasons for this. First, the dFMEA is a bottom-up approach that assumes you know all of the software functions that are needed–but you won’t. Second, the dFMEA will have multiple rows of effects for each failure mode because each cause of software failure can overlap with multiple software functions. Therefore, the best way to analyze data risks is a fault-tree analysis (i.e., FTA). The FTA is the best tool for analysis of software data hazards because you only need three fault trees: 1) Mix-up of data, 2) Loss of data, and 3) Modification of data. In each of these FTAs, all of the potential causes of software failure will be identified in the branches of the fault tree. Analyzing the fault tree structure, specifically the position of OR gates, can assist in software design. OR logic gates that can result in critical failures need additional software risk controls to prevent a single cause of software failure from creating a serious hazardous situation.
Copied from Section 6.2.1.5 from AAMI TIR 80002-1:2009
How to build a fault tree
The first step of your software risk analysis should be to identify data hazards. Once you identify the data hazards, you can build a fault tree for each of the three possible software failures: mix-up of data, 2) incomplete data, and 3) modification of data. Each data hazards can cause multiple software failure modes, but the type of logic gate will determine the outcome of that data hazard. An OR gate will result in software failures if there is just one hazardous event, while an AND gate requires at least two hazardous events to occur before the software failure will occur. The position of the OR/AND gate also impacts the potential for software failures.
In this use error training webinar, you will learn how to use our decision tree form to determine if you have identified a use error or an abnormal use.
Your cart is empty
Use error and abnormal use training webinar ($79)
In this webinar, you will learn what a use error is and what abnormal use is. You will learn how to use our decision tree form to determine if you have identified a use error or an abnormal use. The webinar is a 21-minute recorded video (i.e., mp4) and includes the decision tree form with a work instruction.
Use error and abnormal use training webinar
In this webinar, you will learn what a use error is and what abnormal use is. You will learn how to use our decision tree form to determine if you have identified a use error or an abnormal use. The webinar is a 21-minute recorded video (i.e., mp4) and includes the decision tree form with a work instruction.
Price: $79.00
Please note: These products will be delivered to the email address provided in the shopping cart transaction. After the transaction is verified, please check your email for the download.
When is the use error and abnormal use training webinar?
This webinar is a 21-minute recording you can purchase on-demand and watch the training as often as you wish. If you need help preparing a usability engineering file (UEF) for your device, we can help you on an hourly basis. Please contact Lindsey Walker for a quote.
What you will receive in the Use error and abnormal use training webinar:
Other Usability Engineering / Human Factors Training
Matthew Walker – QMS, Risk Management, Usability Testing, Cybersecurity
Matthew came to us with a regulatory background that focused on OSHA and NFPA regulations when he was a Firefighter/EMT. Since we kidnapped him from his other career, he now works in Medical Device Quality Systems and Regulatory Pathways. He is a Junior in George Washington University’s BSHS- Clinical Research Management Program, and we are proud to say that he is also a member of both the Golden Keys and Phi Theta Kappa Honor Societies! Matthew participates as a member of our audit team and has a passion for risk management and human factors engineering. Always the mad scientist, Matthew pairs his professional life in regulatory affairs with hobbies in the culinary arts as he also holds a Butchers/Meat Cutters certificate from Vermont Technical College.
GSPRs are the General Safety and Performance Requirements for CE Marking of medical devices and IVDs in Annex I of the EU MDR and IVDR.
What are the GSPRs?
General Safety and Performance Requirements (GSPRs) are the requirements for safety and performance specified in Annex I of the EU MDR and EU IVDR. GSPRs are divided into Chapter I (i.e., – Sections 1-9 of the MDR and Sections 1-8 of the IVDR are the General requirements), Chapter II (i.e., – Sections 10-22 of the MDR and Sections 9-19 of the IVD are the Requirements regarding design and manufacture), and Chapter III (i.e., Section 23 of the MDR and Section 20 of the IVDR are the requirements regarding the information supplied with the device or IVD). All devices must meet the requirement of Chapter I and Chapter III, but the applicability of Chapter II depends upon the technological characteristics of the device or IVD.
Where do the GSPRs go in your technical documentation?
When a Notified Body reviews your technical documentation, they expect you to provide either a complete technical file or technical file index that is organized in accordance with Annex II of the MDR or IVDR. Section 4 of Annex II is labeled “General Safety and Performance Requirements.” This section is where your GSPR checklist should be located in the technical file or technical file index. Generally a GSPR checklist is considered the best way to document these requirements. The checklist should should provided traceability to each specific requirement and the following elements:
the applicability or non-applicability of each requirement; justifications shall be documented for non-applicability
the method or methods used to demonstrate conformity with each requirement
the harmonized standards (i.e., EN standard) and/or Common Specifications (CS) applied
identification of controlled documents that provide evidence of conformity with the harmonized standards or CS
What are the subparts of each chapter in the GSPRs?
Chapter I
The general requirements for safety and performance (i.e., Chapter I in Section 1-9 of the MDR and Section 1-8 of the IVDR) are primarily focused on risk management requirements. These first few sections state that the manufacturer must ensure that the device or IVD is safe, effective, and does not compromise the clinical condition or safety of patients or users. The manufacturer must take into account the generally acknowledged state of the art. Risks must be reduced as far as possible without adversely affecting the benefit-risk ratio. The manufacturer must implement a risk management system. Risks associated with use errors shall be eliminated or reduced as far as possible. The characteristics of performance shall not be adversely affected the conditions of use, transport and storage during the lifetime of the device or IVD. Finally, all residual risks shall be minimized and be acceptable when weighted against the benefits to the patients and/or user arising from the intended use during normal conditions of use.
Chapter II of the MDR
This Chapter of the GSPRs is organized into the following subsections of the MDR:
10 – Performance characteristics
11 – Chemical, physical and biological properties
12 – Infection and microbial contamination
13 – Devices incorporating materials of biological origin
14 – Construction of devices and interaction with their environment
15 – Devices with a diagnostic or measuring function
16 – Protection against radiation
17 – Electronic programmable systems
18 – Active devices and devices connected to them
19 – Particular requirements for active implantable devices
20 – Protection against mechanical and thermal risks
21 – Protection against the risks posed to the patient or user by devices supplying energy or substances
22 – Protection against the risks posed by medical devices intended by the manufacturer for use by lay persons
Chapter II of the IVDR
This Chapter of the GSPRs is organized into the following subsections of the IVDR:
9 – Performance characteristics
10 – Chemical, physical and biological properties
11 – Infection and microbial contamination
12 – Devices incorporating materials of biological origin
13 – Construction of devices and interaction with their environment
14 – Devices with a measuring function
15 – Protection against radiation
16 – Electronic programmable systems
17 – Devices connected to or equipped with an energy source
18 – Protection against mechanical and thermal risks
19 – Protection against the risks posed by devices intended for self-testing or near-patient testing
Chapter III
Chapter III is divided into four subparts. Section 23.1 of the MDR and section 20.1 of the IVDR are the general requirements for information provided by the manufacturer (i.e., labeling). The recommended harmonized standard is EN ISO 20417:2021. Section 23.2 of the MDR and section 20.2 of the IVDR include the labeling requirements. Section 23.3 of the MDR and section 20.3 of the IVDR include requirements for information on the packaging which maintains the sterile condition of a device or IVD (i.e., label on the sterile barrier packaging). Finally, Section 23.4 of the MDR and section 20.4 of the IVDR include requirements for the Instructions for Use (i.e., IFU, Directions for Use, or User Manual).
Completing your checklist
Completing the GSPR Checklist would be easy if there were only 20-23 requirements, but most of the requirements have multiple requirements. For example, GSPR 14 of the MDR has 7 subparts, 18 of the MDR has 8 subparts, and labeling requirements are six pages long. Each subpart must be addressed when you complete the columns of the checklist. If any of the parts or subparts do not apply to your device, you need to provide a justification. When you write your justification for the non-applicability of a GSPR, you need to be careful to provide a justification for each subpart of the requirement–even if the subpart is not separately identified by a letter or number.
Download our Checklist
If you need a template for creating your own GSPR checklist, you can download our template by filling in the form below:
How do you address differences from the Essential Principles of Safety and Performance?
Health Canada also identifies Essential Principles for Safety and Effectiveness in Sections 10-20 of the Canadian Medical Device Regulations (i.e., SOR 98/282) that is similar to the European GSPRs, and Australia has a similar Essential Principles Checklist document with only a few minor differences. The Global Harmonized Task Force (GHTF) created an earlier version in 2005, but the International Medical Device Regulators Forum (IMDRF) released a newer version in 2024. Health Canada will typically accept your GSPR checklist developed for CE Marking, but a gap analysis should be performed against the Australian Regulations.
As the Marketing Lead at BW&CO, Alexina holds a Bachelor of Business Administration in Marketing from the University of Houston. With a deep understanding of the grant funding landscape, she drives strategic marketing initiatives that elevate BW&CO’s brand and reach. Fluent in both English and French, Alexina excels at creating and executing campaigns that connect with diverse audiences. Her expertise includes developing content strategies, managing digital marketing efforts, and conducting market research to support business growth.
