510(k) Archives

Archive for 510(k)

Artificial Intelligence and Machine Learning Medical Devices

Posted by Mary Vater on April 11, 2023

The FDA released a new draft guidance document about artificial intelligence and machine learning (AI/ML) functions in medical devices.

What is a predetermined change control plan for artificial intelligence (AI) software?

The new FDA guidance is specific to predetermined change control plans for marketing submissions. The guidance was released on March 30, 2023, but the document is dated April 3, 2023. The draft guidance applies to artificial intelligence (AI) or Machine Learning-Enabled Device Software Functions (ML-DSF), including modifications automatically implemented by the software and modifications to the models implemented manually.

New Artificial Intelligence PCCP Guidance Document 1024x857 Artificial Intelligence and Machine Learning Medical Devices

A PCCP must be authorized through 510k, De Novo, or PMA pathways, as appropriate. The purpose of including a PCCP in a marketing submission is to seek premarket authorization for these intended device modifications without necessitating additional marketing submissions for each change described in the PCCP.

How do you determine if a 510k is required for a device modification, and how would a PCCP affect this?

Currently, there are three guidance documents relating to the evaluation of changes and determination if a new premarket submission is required:

Deciding When to Submit a 510k for a Software Change to an Existing Device
Deciding When to Submit a 510k for a Change to an Existing Device
Modifications to Devices Subject to Premarket Approval (PMA) – The PMA Supplement Decision-Making Process

These guidance documents will still be the first steps in evaluating changes. Only changes specific to artificial intelligence (AI) or ML-DSF that would result in a new pre-market submission could be subject to a PCCP.

Examples of Employing AI/ML-DSF PCCPs

Retraining a model with more data to improve device performance while maintaining or increasing sensitivity. If this type of change is pre-approved in the PCCP, the labeling can be updated to reflect the improved performance once the change has been implemented.
Extending the scope of compatible hardware with a device system. For example, if the algorithm was initially trained using one specific camera, ultrasound, defined parameter, etc., then a PCCP could add additional cameras/ultrasounds/modified parameters.
Retraining a model to optimize site-specific performance for a specific subset of patients with a particular condition for whom sufficient data was unavailable. The PCCP could expand the indications once such data were available.

What is the difference between a locked vs. adaptive algorithm?

A locked algorithm is a software function involving human input, action, review, and/or decision-making before implementation. Once the algorithm is designed and implemented, it cannot be changed without modifying the source code.

Locked algorithms contrast with adaptive/automatic algorithms, where the software will implement changes without human intervention. The adaptive/automatic algorithms are designed to adjust according to changing input conditions. The adaptive/automatic algorithm is designed to recognize patterns in the input data and adjust its processing accordingly.

Typically locked algorithms apply to fixed functions such as a decision tree, static look-up table, or complex classifier. For AI/ML-DSF, manually implemented algorithms may involve training the algorithm on a new dataset or serving a new function. Once the training is complete, the algorithm will be implemented into the software. Adaptive algorithms are programmed such that their behavior changes over time as it is run based on the information it processes.

As it relates to a PCCP, the detailed description of the intended modifications needs to specify which algorithm type is being modified.

What is included in a PCCP for artificial intelligence (AI) software?

A PCCP should consist of:

Detailed Description of Intended Modifications
Modification Protocol describing the verification and validation activities, including pre-defined acceptance criteria
Impact Assessment identifying the benefits and risks introduced by the changes

The detailed description of the intended modifications should list each proposed device modification and the rationale for each change. If changes require labeling modifications, that should also be described. It should also be clearly stated whether or not the proposed change is intended to be implemented automatically or manually. The description should describe whether the change will be implemented globally across all devices on the market or locally, specific to different devices based on the unique characteristics of the device’s patient or clinical site.

The types of modifications that are appropriate for a PCCP include modifications related to quantitative measurements of ML-DSF performance specifications, changes related to device inputs, and limited modifications relating to the device’s use and performance. The draft guidance provides some examples of each of those modification types.

The content of the modification protocol section requires a description of planned data management practices relating to the reference standard and annotation process, a description of re-training practices and processing steps, performance evaluation methods and acceptance criteria, and internal procedures for implementing updates.

The impact assessment is the documentation of the evaluation of the benefits and risks of implementing the PCCP for the software. Any controls or mitigations of the risks should be described in this section.

Appendix A of the draft guidance includes example elements of modification protocol components for ML-DSFs. Appendix B includes examples of ML-DSF scenarios employing PCCPs.

If, at some point, the manufacturer wants to make changes to the content of the PCCP relating to either the modifications described or the methods used to validate those changes, that generally would require a new marketing submission for the device.

Utilizing a PCCP in your QMS Change Control System

When evaluating and implementing changes, the manufacturer shall do so in accordance with their Quality Management System change control processes. This should require a review of planned modifications against the FDA guidance documents for evaluating changes and the PCCP. For the change to be acceptable under the PCCP, it must be specified in the Description of Modifications and implemented in conformance with the methods and specifications described in the Modification Protocol. A new premarket submission is required if it does not meet those requirements.

Posted in: 510(k), Artificial Intelligence (AI) & Machine Learning (ML), Design Change Control

Regulatory pathway analysis–a case study

Posted by Rob Packard on March 22, 2023

This article uses a case study example to explain how to determine the correct regulatory pathway for your medical device through the US FDA.

Regulatory Pathway 1 Regulatory pathway analysis a case study

How do you select the right regulatory pathway for your device?

Every consultant likes to answer this type of question with the answer, “It depends.” Well, of course, it depends. If there was only one answer, you could google that question, and you wouldn’t need to pay a regulatory consultant to answer the question. A more useful response is to start by asking five qualifying questions:

Does your product meet the definition of a device?
What is the intended purpose of your product?
How many people in the USA need your product annually?
Is there a similar product already on the market?
What are the risks associated with your product?

The first question is important because some products are not regulated as medical devices. If your product does not diagnose, treat, or monitor a medical condition, then your product may not be a device. For example, the product might be considered a general wellness product or clinical decision support software. In addition, some products have a systemic mode of action, and these products are typically categorized as a drug rather than a device–even if the product includes a needle and syringe.

The intended purpose of a product is the primary method used by the US FDA to determine how a product is regulated. This also determines which group within the FDA is responsible for reviewing a submission for your product. The US regulations use the term “intended use” of a device, but the decision is based upon the “indications for use” which are more specific. To understand the difference, we created a video explaining the difference.

Even regulatory consultants sometimes forget to ask how many people need your product annually, but population size determines the regulatory pathway. Any intended patient population less than 8,000 patients annually in the USA is eligible for a humanitarian device exemption with a special regulatory pathway and pricing constraints. If your product is intended for a population of <8,000 people annually, your device could qualify for a humanitarian device exemption, and the market is small enough that there may not be any similar products on the market.

If similar products are already on the US market, determining the regulatory pathway is much easier. We can look up the competitor product(s) in the FDA’s registration and listing database. In most cases, you must follow the same pathway your competitors took, and the FDA database will tell us your regulatory pathway.

If all of the products on the US market have different indications for use, or the technological characteristics of your product are different from other devices, then you need to categorize your product’s risks. For low-risk devices, general controls may be adequate. For medium-risk devices, special controls are required by the FDA. For the highest-risk devices, the FDA usually requires a clinical study, a panel review of your clinical data, and the FDA requires pre-market approval.

This article will use the example of bipolar forceps used with an electrosurgical generator as a case study.

Bipolar Forceps Regulatory pathway analysis a case study

What is the US FDA regulatory pathway for you device?

The generic term used for regulator authorization is “approval,” but the US FDA reserves this term for Class 3 devices with a Premarket Approval (PMA) submission . The reason for this is that only these submissions include a panel review of clinical data to support safety and effectiveness of the device. Approval is limited to ~30 devices each year, and approximately 1,000 devices have been approved through the PMA process since 1976 when the US FDA first began regulating medical devices.

Most Class 2 devices are submitted to the FDA as Premarket Notifications or 510k submissions. This process is referred to as “510k clearance,” because clinical data is usually not required with this submissions and there is no panel review of safety and effectiveness data. A 510k was originally planned as a rare pathway that would only be used by devices that are copies of other devices that are already sold on the market. However, the 510k pathway became the defacto regulatory pathway for 95+% of devices that are sold in the USA.

For moderate and high-risk devices that are intended for rare patient populations (i.e., <8,000 patients per year in the USA), the humanitarian device exemption process is the regulatory pathway.

Class 1 devices typically do not require a 510k submission, most of these devices are exempt from design controls, and some are exempt from quality system requirements. These devices still require listing on the FDA registration and listing database, but there is no review of the device by the FDA to ensure you have correctly classified and labeled Class 1 devices.

How do you find a predicate for your 510k submission?

As stated above, one of the most critical questions is, “Is there a similar product already on the market?” For our example of bipolar forceps, the answer is “yes.” There are approximately 169 bipolar forceps that have been 510k cleared by the FDA since 1976. If you are developing new bipolar forceps, you must prepare a 510k submission. The first step of this process is to verify that a 510k submission is the correct pathway and to find a suitable competitor product to use as a “predicate” device. A predicate device is a device that meets each of the following criteria:

it is legally marketing in the USA
it has indications for use that are equivalent to your device
the technological characteristics are equivalent to your device

There are two search strategies we use to verify the product classification of a new device and to find a suitable predicate device. The first strategy is to use the free, public databases provided by the FDA. Ideally, you instantly think of a direct competitor that sells bipolar forceps for electrosurgery in the USA (e.g., Conmed bipolar forceps). You can use the registration and listing database to find a suitable predicate in this situation. First, you type “Conmed” into the database search tool for the name of the company, and then you type “bipolar forceps” in the data search tool for the name of the device.

Registration and Listing for Conmed Bipolar Forceps 1024x443 Regulatory pathway analysis a case study

If you are unaware of any competitor products, you will need to search using the product classification database instead. Unfortunately, this approach will result in no results if you use the terms “bipolar” or “forceps.” Therefore, you will need to be more creative and use the word “electrosurgical,” which describes a larger product classification that includes both monopolar and bipolar surgical devices that have many sizes and shapes–including bipolar forceps. The correct product classification is seventh out of 31 search results.

GEI Product code 1024x454 Regulatory pathway analysis a case study

Listing for Conmed Specification Developer 1024x398 Regulatory pathway analysis a case study

The most significant disadvantage of the FDA databases is that you can only search each database separately. The search is also a boolean-type search rather than using natural language algorithms that we all take for granted. The second strategy is to use a licensed database (e.g., Basil Systems).

Searching these databases is more efficient, and the software will provide additional information that the FDA website does not offer, such as a predicate tree, review time, and models listed under each 510k number are provided below:

Predicate Tree for K190909 1024x539 Regulatory pathway analysis a case study

What does the predicate tree look like for the predicate device you selected?

Review Time for devices in the GEI product classification code 1024x452 Regulatory pathway analysis a case study

I’m glad I don’t need to manually enter the 510k review time for 2,263 devices to create the above graph.

Conmed bipolar forceps listed under K854864 1024x323 Regulatory pathway analysis a case study

Wouldn’t having the model numbers for every device identified in the US FDA listing database be nice?

Another advantage of the Basil Systems software is that the database is lightning-fast, while the FDA is a free government database (i.e., not quite as fast).

How do you create a regulatory pathway strategy for medical devices?

The best strategy for obtaining 510k clearance is to select a predicate device with the same indications for use that you want and was recently cleared by the FDA. Therefore, you will need to review FDA Form 3881 for each of the potential predicate devices you find for your device. In the case of the bipolar forceps, there are 169 devices to choose from, but FDA Form 3881 is not available for 100% of those devices because the FDA database only displays FDA Form 3881 and the 510(k) Summary for devices cleared since 1996. Therefore, you should select a device cleared by the FDA in the past ten years unless there are no equivalent devices with a recent clearance.

K190909 FDA Form 3881 798x1024 Regulatory pathway analysis a case study

In addition to identifying the correct product classification code for your device and selecting a predicate device, you will also need to develop a testing plan for the verification and validation of your device. For electrosurgical devices, there is an FDA special controls guidance that defines the testing requirements and the content required for a 510k submission. Once you develop a testing plan, you should confirm that the FDA agrees with your regulatory strategy and testing plan in a pre-submission meeting.

Which type of 510k submission is required for your device?

There are three types of 510k submissions:

Special 510k – 30-day review target timeline
Abbreviated 510k – 90-day review target timeline (requires summary reports and use of recognized consensus standards)
Traditional 510k – 90-day review target timeline

The special 510k pathway is intended for minor device modifications from the predicate device. However, this pathway is only eligible to your company if your company also submitted the predicate device. Originally it was only permitted to submit a Special 510k for modifications that require the review of one functional area. However, the FDA recently completed a pilot study evaluating if more than one functional area could be reviewed. The FDA determined that up to three functional areas could be reviewed. However, the FDA decides whether they can complete the review within 30 days or if you need to convert your Special 510k submission to a Traditional submission. Therefore, you should also discuss the submission type with the FDA in a pre-submission meeting if you are unsure whether the device modifications will allow the FDA to complete the review in 30 days.

In 2019 the FDA updated the guidance document for Abbreviated 510k submissions. However, this pathway requires that the manufacturer use recognized consensus standards for the testing, and the manufacturer must provide a summary document for each test report. The theory is that abbreviated reports require less time for the FDA to review than full test reports. However, if you do not provide sufficient information in the summary document, the FDA will place your submission on hold and request additional information. This happens for nearly 100% of abbreviated 510k submissions. Therefore, there is no clear benefit for manufacturers to take the time to write a summary for each report in the 510k submission. This also explains why less than 2% of submissions were abbreviated type in 2022.

The traditional type of 510k is the most common type of 510k submission used by manufacturers, and this is the type we recommend for all new device manufacturers.

Tags: 510(k), FDA, Medical Device, Regulatory Pathway

Posted in: 510(k), FDA

Leave a Comment (0) →

Four easy ways 510k and De Novo content is different

Posted by Rob Packard on January 17, 2023

It’s a common misconception that FDA De Novo content is very different from FDA 510k submission content, but is that true?

What do you think the De Novo content differences are?

Most people think the difference between a 510k and a De Novo is time and money. That conclusion is based upon a very important assumption: a 510k will not require clinical data, and a De Novo will require clinical data. That assumption is not always correct. 10-15% of 510k submissions include clinical data to support the performance claims, and last year our team submitted three De Novo submissions that did not include any clinical data. So what are the differences between a 510k and a De Novo content?

We use the same FDA eSTAR template for both types of FDA submissions, and on the first page of the eSTAR template, we identify if the submission is a 510k or De Novo. If we select De Novo, the eSTAR will be pre-populated with four unique De Novo content requirements that are not found in a 510k. The four unique requirements are:

identifying alternative practices and procedures for the same indications
recommending a classification, providing a justification for that classification, and explaining what efforts were taken to identify a suitable 510k product code
providing a written benefit/risk analysis starting with the clinical benefits of your device
recommendations for special controls for your new product code based upon the risks to health and the mitigation measures for each risk

Alternate practices and procedures 1024x547 Four easy ways 510k and De Novo content is different

What alternative practices and procedures are currently available?

The unique De Novo content requirement is to provide a description of alternative practices and procedures for treatment or diagnosis of the same indications that you are proposing for your subject device. This is a subsection of the device description section in the FDA eSTAR template. Your should description should include other 510k-cleared products, drugs, and even products that have similar indications but are not identical. The description of alternative practices and procedures must also be attached as a document in the section for benefits, risks, and mitigation measures. To maintain consistency throughout your submission, you should create the document for attachment first and copy and paste the content into the text box at the end of the device description section.

You need to recommend a classification in your De Novo

The unique De Novo content requirement is found in a section titled “Classification.” There is a shorter classification section included in 510k submissions, but the 510k version only has four cells. The first three are populated by selecting one of the options from a dropdown menu, and the fourth cell is only used if your subject device includes other product classification codes.

Classification 1024x346 Four easy ways 510k and De Novo content is different

The De Novo version of the eSTAR is identical for the first row of the classification section, but then you must select a proposed product classification (i.e., Class 1 or Class 2) in accordance with FDA Classification Procedures (i.e., 21 CFR 860). The third cell is a text box for you to enter your justification for the proposed classification. Next, the FDA requires you to enter a proposed classification name. Finally, at the end of the classification section, the FDA requires that you provide a classification summary or reference to a previous NSE 510k submission.

A Benefit/Risk Analysis is required in the De Novo Content

For new devices, the FDA uses a benefit/risk analysis to decide if a device should be authorized for marketing in the USA. This process includes humanitarian device exemptions, De Novo applications, and Premarket Approval submissions. The FDA has a guidance document that provides guidance for FDA reviewers and the industry. The most important aspect is, to begin with, the benefits of the device and to provide a quantitative comparison of benefits and risks. Many De Novo submissions have been rejected because the submitter did not provide objective evidence of clinical benefits for the subject device.

Benefit Risk Analysis 1024x210 Four easy ways 510k and De Novo content is different

The FDA guidance documents are helpful for creating a benefit/risk analysis, but you can also find information in the ISO/TR 24971:2020–the guidance for the application of ISO 14971:2019. Our company also includes a template for a benefit/risk analysis as part of our risk management procedure (i.e., SYS-010).

What are your recommended Special Controls?

In FDA De Novo Classification Decision Summaries, there is a table provided that identifies the identified risks to health and the recommended mitigation measures for each risk category. In the FDA eSTAR, you are required to add a similar table for De Novo content. The only difference between the table in summary and the eSTAR is that the eSTAR table has a third column where the FDA wants you to reference the supporting data provided for each mitigation measure–including the document and page within the document. The FDA also provided an example table in the eSTAR, copied below.

Risk Mitigation Table Four easy ways 510k and De Novo content is different

The above table for the risks to health and mitigations needs to be translated into a list of recommended Special Controls for Class II devices. Since most De Novo applications are for Class II devices, you will need to convert each of your mitigations into a corresponding Special Control and type these controls into the text box provided in the FDA eSTAR.

Special Controls Four easy ways 510k and De Novo content is different

What else is different from a 510k?

There are no additional mandatory elements that you need to include in a De Novo application, but there are several elements of a 510k submission that are not included in a De Novo. The most obvious of these sections is the Substantial Equivalence Comparison Table in the section labeled “Predicates and Substantial Equivalence.” Another difference is that you are more likely to need clinical data to support a De Novo application than for a 510k submission. It is also possible that subsequent 510k submissions for the same product code may not need to provide clinical data because the 510k process only requires a demonstration of substantial equivalence rather than clinical benefits outweighing risks to health. The FDA review time for a Traditional 510(k) varied between 190 and 210 days in 2022, while the De Novo review timeline averaged 390 days in 2022. Finally, the FDA user fees for 510k submissions are far less than those for a De Novo application.

Posted in: 510(k), De Novo

Leave a Comment (0) →

The best human factors questions in every successful FDA meeting are?

Posted by Rob Packard on January 4, 2023

What are the best human factors questions to ask the FDA during a pre-submission meeting, and what information content do you need in a 510k?

Talk to the FDA before human factors validation

The FDA did not start enforcing the requirement to apply human factors and usability engineering to medical device design until 2017 because the final version of the human factors guidance document was not released until February 3, 2016. Approximately ninety percent of the human factors testing reports submitted to the FDA in 510k pre-market submissions are deficient because the 510k submission content only includes the final summative testing report. The FDA needs a complete usability engineering file, and the human factors information needs to comply with FDA guidelines for the format and content of a 510k pre-market submission–not just IEC 62366-1:2015.

What human factors information does the FDA want?

For several years, FDA submission deficiency letters indicated that you should not include the frequency of occurrence in your estimation of use-related risks, but the FDA never provided this information in a guidance document. On December 9, 2022, the FDA finally released a draft human factors guidance regarding the format and content of a 510k pre-market submission. The new draft guidance includes the requirement for a use-related risk analysis (URRA) in table 2 (copied below).

Table 2 example of tabular format for the URRA 1024x354 The best human factors questions in every successful FDA meeting are?

In this new draft FDA guidance, the FDA identifies three different human factors submission categories. For the first category, only a conclusion and high-level summary are needed. For the second category, a user specification is also needed. For the third category, you need a comprehensive human factors engineering report with the following elements described in Section IV of the draft FDA guidance:

Submission Category 1, 2, and 3

Conclusion and high-level summary

Submission Category 2 and 3

Descriptions of intended device users, uses, use environments, and training
Description of the device-user interface
Summary of known use problems

Submission Category 3 only

Summary of preliminary analyses and evaluations
Use-related risk analysis to analyze hazards and risks associated with the use of the device
Identification and description of critical tasks
Details of validation testing of the final design

Before you spend tens of thousands or hundreds of thousands of dollars on human factors testing, you want to make sure the FDA agrees with your human factors testing plan. Otherwise, you will pay for the testing twice: once for your initial submission and a second time in your response to the FDA request for additional information to address deficiencies. Testing can cost more than your electrical safety testing. The facility needs to have the right equipment and space for the testing, you need support personnel to set up the equipment, you need to recruit participants, you need to compensate participants, and you need device samples.

When can you ask the FDA human factors questions?

The FDA cannot provide consulting advice on a submission, and the agency will not review data during pre-submission meetings. The FDA can provide feedback on protocols, specifications, and scientific justifications. Therefore, you should submit questions to the FDA in a pre-submission when you have a draft protocol, a draft specification, or a draft justification for why a task is not critical. Pre-submissions are “non-binding.” You can change your design and approach to human factors. Therefore, don’t wait until your information is 100% finalized. Share your documentation at the draft stage during the development phase and before your design freeze. You need these answers when you are planning a study and obtaining quotes.

What are the best human factors questions to ask in a pre-sub?

In the FDA guidance for pre-submission meetings, the FDA provides suggested questions to ask:

Does the Agency have comments on our proposed human factors engineering process?
Is the attached use-related risk analysis plan adequate? Does the Agency agree that we have identified all the critical tasks?
Does the Agency agree with our proposed test participant recruitment plan for the human factors validation testing?

The above examples are only suggestions, but the best approach is to provide a brief example of what the human factors information will look like and ask the FDA to comment on the examples. The FDA does not have time to review data during a pre-sub meeting, but the FDA can review a few rows extracted from your URRA and comment on your proposed approach to the human factors process.

Human factors questions that are not appropriate

The FDA pre-submission guidance cautions you only to ask 3-4 questions for each meeting request because the FDA has difficulty answering more questions in a 60-minute teleconference. Therefore, you should not ask questions already answered in the FDA guidance. The new draft guidance includes examples of when a device modification can leverage existing human factors information and when new information is needed to support a premarket submission. Instead of asking a question specific to leveraging existing human factors information, instead, provide your rationale for leveraging existing data and ask if the FDA has any concerns with your overall approach to human factors.

Recommended human factors action items

Create a procedure for your human factors process that includes detailed instructions for creating the information required in a usability engineering report and templates for each document.

Tags: human factors

Posted in: 510(k), FDA, Human Factors, Usability

Software validation documentation for a medical device

Posted by Rob Packard on November 29, 2022

Learn why you need to start with software validation documentation before you jump into software development.

When do you create software validation documentation for a medical device or IVD?

At least once a week, I speak with the founder of a new MedTech company that developed a new software application as a medical device (SaMD). The founder will ask me to explain the process for obtaining a 510(k), and they want help with software validation documentation. Many people I speak with have never even heard of IEC 62304.

Even though they already have a working application, usually, validation documentation has not even been started. Although you can create all of your software validation documentation after you create a working application, certain tasks are important to perform before you develop software code. Jumping into software development without the foundational documentation will not get your device to market faster. Instead, you will struggle to create documentation retroactively, and the process will be slower. In the end, the result will be a frustrating delay in the launch of your device.

What are the 11 software validation documents required by the FDA?

In 2005 the FDA released a guidance document outlining software validation documentation content required for a premarket submission. There were 11 documents identified in that guidance:

software validation documentation 1024x385 Software validation documentation for a medical device

What the FDA guidance fails to explain is that some of these documents need to be created before software development begins, or your software validation documentation will be missing critical design elements. Therefore, it is important to create a software development plan that schedules activities that result in those documents at the right time. In contrast, four of the eleven documents can wait until your software development is complete.

Which of the software validation documents can wait until the end?

The level of concern only determines what documents the FDA wants to review in a submission rather than what documents are needed for a design history file. In fact, the level of concern (LOC) document is no longer required as a separate document in premarket submissions using the FDA eSTAR template because the template already incorporates the questions that document your LOC. The revision level history document is simply a summary of revisions made to the software during the development process, and that document can be created manually or automatically at the end of the process, or the revision level history can be a living document that is created as changes are made. The traceability matrix can also be a living document created as changes are made, but its only purpose is to act as a tool to provide traceability from hazards to software requirements, to design specifications, and finally to verification and validation reports. Other software tools, such as Application Lifecycle Management (ALM) Software, are designed to ensure the traceability of every hazard and requirement throughout the entire development process. Finally, unresolved anomalies should only be documented at the time of submission. The list may be incomplete until all verification and validation testing is completed, and the list should be the shortest at the time of submission.

What documentation will be created near the end of development?

The software design specification (SDS) is typically a living document until your development process is completed, and you may need to update the SDS after the initial software release to add new features, maintain interoperability with software accessories, or change security controls. The SDS can not begin, however, until you have software requirements and the basic architecture defined. The verification and validation activities are discrete documents created after each revision of the SDS and must therefore be one of the last documents created–especially when provided to the FDA as a summary of the verification and validation efforts.

Which validation documents do you need first?

At the beginning of software development, you need a procedure(s) that defines your software development process. That procedure should have a section that explains the software development environment–including how patches and upgrades will be controlled and released. If you don’t have a quality system procedure that defines your development process, then each developer may document their coding and validation activities differently. That does not mean that you can’t improve or change the procedure once development has begun, but we recommend limiting the implementation of a revised procedure when making major software changes and discussing how revisions will be implemented for any work that remains in progress or has already been completed.

When do the remaining software validation documents get created?

The remaining four software validation documents required for a premarket submission to the FDA are:

Software description
Software hazard analysis
Software requirements specification (SRS)
Architecture design chart

Your development process will be iterative, and therefore, you should be building and refining these four documents iteratively in parallel with your software code. At the beginning of your project, your design plan will need a brief software description. Your initial software description needs to include the indications for use, a list of the software’s functional elements, and the elements of your user specification (i.e., intended patient population, intended users, and user interface). If you are using lean startup methodology, the first version of your device description will be limited to a minimal viable product (MVP). The target performance of the MVP should be documented as an initial software requirements specification (SRS). This initial SRS might only consist of one requirement, but the SRS will expand quickly. Next, you need to perform an initial software hazard analysis to identify the possible hazards. It is important to remember that software hazards are typically hazardous situations and are not limited to direct physical harm. For each potential hazard you identify in your hazard analysis, you will need a software requirement to address each hazard, and each requirement needs to be added to your SRS. As your software becomes more complex by adding software features, your device description needs to be updated. As you add functions and requirements to your software application, your SRS will need updates too. Finally, your development team will need a tool to track data flow and calculations from one software function to the next. That tool is your architecture design chart, and you will want to organize your SRS to match the various software modules identified in your architecture diagram. This phase is iterative and non-linear, you will always have failures, and typically a team of developers will collaborate virtually. Maintaining a current version of the four software documents is critical to keeping your development team on track.

How do you perform a software hazard analysis?

One of the most important pre-requisite tasks for software developers is conducting a hazard analysis. You can develop an algorithm before you write any code, but if you start developing your application to execute an algorithm before you perform a software hazard analysis, you will be missing critical software requirements. Software hazard analysis is different from traditional device hazard analysis because software hazards are unique to software. A traditional device hazard analysis consists of three steps: 1) answering the 37 questions in Annex A of ISO/TR 24971:2020, 2) systematically identifying hazards by using Table C1 in Annex C of ISO 14971:2019, and 3) reviewing the risks associated with previous versions of the device and similar competitor devices. A software hazard analysis will have very few hazards identified from steps 1 and 2 above. Instead, the best resource for software hazard analysis is IEC/TR 80002-1:2009. You should still use the other two standards, especially if you are developing software in a medical device (SiMD) or firmware, but IEC/TR 80002-1 has a wealth of tables that can be used to populate your initial hazards analysis and to update your hazard analysis when you add new features.

How do you document your hazard analysis?

Another key difference between a traditional hazard analysis and a software hazard analysis is how you document the hazards. Most devices use a design FMEA (dFMEA) to document hazards. The dFMEA is a bottom-up method for documenting your risk analysis by starting with device failure modes. Another tool for documenting hazards is a fault tree diagram.

Fault Tree Example from AAMI TIR 80002 1 2009 300x239 Software validation documentation for a medical device

Copied from Section 6.2.1.5 from AAMI / IEC TIR 80002-1:2009

A fault tree is a top-down method for documenting your risk analysis, where you identify all of the potential causes that contribute to a specific failure mode. Fault tree diagrams lend themselves to complaint investigations because complaint investigations begin with the identification of the failure (i.e., complaint) at the top of the diagram. For software, the FDA will not allow you to use the probability of occurrence to estimate risks. Instead, software risk estimation should be limited to the severity of the potential harm. Therefore, a fault tree diagram is generally a better tool for documenting software risk analysis and organizing your list of hazards. You might even consider creating a separate fault tree diagram for each module of your software identified in the architecture diagram. This approach will also help you identify the potential impact of any software hazard by looking at the failure at the top of the fault tree. The higher the potential severity of the software failure, the more resources the software team needs to apply to developing software risk controls and verifying risk control effectiveness for the associated fault tree.

Tags: 510(k), fault tree, hazard analysis, IEC 62304, IEC/TR 80002-1, premarket submission, SaMD, SiMD, software validation documentation

Posted in: 510(k), FDA, Software Verification and Validation

Leave a Comment (0) →

510k Electronic Submission Guidance for FDA 510k Submissions

Posted by Rob Packard on November 8, 2022

This article provides an overview of the new 510k electronic submission guidance document that the FDA released in September 2022.

What’s included in the 510k electronic submission guidance?

As with any FDA guidance, there is an introduction and background regarding the reason for the new 510k electronic submission guidance document (i.e., eSTAR guidance). In the scope section, the FDA specifies that this document is specific to 510k submissions using the eSTAR template. The FDA has plans to release a similar De Novo submission guidance for using the eSTAR template. In the “Significant Terminology” section of the guidance, the FDA provides definitions for each of the different types of submissions: eCopy, eSubmitter, etc. Then the next to last section of the electronic submission guidance, the FDA provides a table outlining all of the sections of the new eSTAR template. The table is reproduced later in this article. If you are interested in a tutorial on completing each section outlined in the table, we recommend purchasing Medical Device Academy’s 510(k) Course. The last section of the eSTAR guidance indicates the timing for compliance with the new guidance.

What is the deadline for compliance with the guidance?

In this new 510k electronic submission guidance document, the FDA indicates that using the eSTAR template and compliance with the eSTAR guidance is voluntary until October 1, 2023. Therefore, after FY 2023, all 510k submissions must be submitted as an eSTAR submission instead of using the older traditional 510k submission format. The FDA also provides a hyperlink for requesting a Customer Collaboration Portal (CCP) account.

What’s missing from this 510k submission guidance?

The new 510k electronic submission guidance does not provide information regarding the receipt date for electronic submissions made through the new customer collaboration portal (CCP) created by CDRH. The image below is a screen capture of the current CCP upload webpage. It includes the following statement, “Send your submission before 16:00 ET on a business day for us to process it the same day.” This statement was added sometime in August or September, but the FDA has not released a detailed explanation. This statement makes it clear that the FDA is not promising to process a submission the “same day” if the submission is received after 4:00 pm ET. However, “processed” does not have the same meaning as “receipt date.”

Another element missing from this new guidance is a reference to human factors documentation. For any devices that have a user interface that is different from the predicate device, and for software devices, the FDA requires documentation of your human factors process to make sure that differences in the user interface do not result in new or different risks when compared to the predicate device. The 2016 FDA guidance for human factors has not been updated, but FDA reviewers continue to issue deficiencies related to the objective evidence provided in a 510k for human factors validation.

CCP screen capture 1024x619 510k Electronic Submission Guidance for FDA 510k Submissions

The FDA must be consistent in the wording for “Hours for Receipt of Submission” because this affects submissions at the end of the fiscal year, but it also affects any submissions with a deadline for response to an RTA Hold, AI Response, and IDE submissions. The CDER and CBER divisions of the FDA address the need for defining the date of receipt in a guidance document specific to this topic, “Providing Regulatory Submissions in Electronic Format–Receipt Date.” Below is a screen capture copied from page 4 of the guidance.

Electronic Submission 510k Electronic Submission Guidance for FDA 510k Submissions

What are the new sections for a 510k submission?

In 2019, the FDA released a guidance document on the “Format of Traditional and Abbreviated 510(k)s.” That guidance outlines the 20 sections of a traditional 510k submission that have been used for decades. However, the new 510k electronic submission guidance has no numbering for the sections of the eSTAR template, and there are 22 sections instead of 20 sections. Several of the new sections are elements of the current FDA submission cover sheet (i.e., FDA Form 3514), and some sections exist in the 2019 guidance that were eliminated, such as: “Class III Summary and Certification.” Therefore, Medical Device Academy is recreating 100% of our 510k training webinars to explain how our 510k templates are used with the 510k eSTAR template and how to fill in the PDF form. To prevent confusion between the two formats, we are using letters for each section in the eSTAR template instead of numbers (i.e., A-V instead of 1-20). Table 1 from the new eSTAR guidance is reproduced below for your information.

	Information Requested	Description
A	Submission Type	Identification of key information that may be useful to FDA in the initial processing and review of the 510(k) submission, including content from current Form FDA 3514, Section A.
B	Cover Letter / Letters of Reference	Attach a cover letter and any documents that refer to other submissions.
C	Submitter Information	Information on submitter and correspondent, if applicable, consistent with content from current Form FDA 3514, Sections B and C.
D	Pre-Submission Correspondence & Previous Regulator Interaction	Information on prior submissions for the same device included in the current submission, such as submission numbers for a prior not substantially equivalent (NSE) determination, prior deleted or withdrawn 510(k), Q-Submission, Investigational Device Exemption (IDE) application, premarket approval (PMA) application, humanitarian device exemption (HDE) application, or De Novo classification request.
E	Consensus Standards	Identification of voluntary consensus standard(s) used, if applicable. This includes both FDA-recognized and nonrecognized consensus standards.
F	Device Description	Identification of listing number if listed with FDA. Descriptive information for the device, including a description of the technological characteristics of the device including materials, design, energy source, and other device features, as defined in section 513(i)(1)(B) of the FD&C Act and 21 CFR 807.100(b)(2)(ii)(A). Descriptive information also includes a description of the principle of operation for achieving the intended effect and the proposed conditions of use, such as surgical technique for implants; anatomical location of use; user interface; how the device interacts with other devices; and/or how the device interacts with the patient. Information on whether the device is intended to be marketed with accessories. Identification of any applicable device-specific guidance document(s) or special controls for the device type as provided in a special controls document (or alternative measures identified that provide at least an equivalent assurance of safety and effectiveness) or in a device-specific classification regulation, and/or performance standards. See “The 510(k) Program: Evaluating Substantial Equivalence in Premarket Notifications [510(k)].”
G	Proposed Indications for Use (Form FDA 3881)	Identification of the proposed indications for use of the device. The term indications for use, as defined in 21 CFR 814.20(b)(3)(i), describes the disease or condition the device will diagnose, treat, prevent, cure, or mitigate, including a description of the patient population for which the device is intended.
H	Classification	Identification of the classification regulation number that seems most appropriate for the subject device, as applicable.
I	Predicates and Substantial Equivalence	Identification of a predicate device (e.g., 510(k) number, De Novo number, reclassified PMA number, classification regulation reference, if exempt and limitations to exemption are exceeded, or statement that the predicate is a preamendments device). The submission should include a comparison of the predicate and subject device and a discussion why any differences between the subject and predicate do not impact safety and effectiveness [see section 513(i)(1)(A) of the FD&C Act and 21 CFR 807.87(f)]. A reference device should also be included in the discussion, if applicable. See “The 510(k) Program: Evaluating Substantial Equivalence in Premarket Notifications [510(k)].”
J	Design/Special Controls, Risks to Health, and Mitigation Measures	Applicable to Special 510(k) submissions only. Identification of the device changes and the risk analysis method(s) used to assess the impact of the change(s) on the device and the results of the analysis. Risk control measures to mitigate identified risks (e.g., labeling, verification). See “The Special 510(k) Program.”
K	Labeling	Submission of proposed labeling in sufficient detail to satisfy the requirements of 21 CFR 807.87(e). Generally, if the device is an in vitro diagnostic device, the labeling must also satisfy the requirements of 21 CFR 809.10. Additionally, the term “labeling” generally includes the device label, instructions for use, and any patient labeling. See “Guidance on Medical Device Patient Labeling.”
L	Reprocessing	Information for assessing the reprocessing validation and labeling, if applicable. See “Reprocessing Medical Devices in Health Care Settings: Validation Methods and Labeling.”
M	Sterility	Information on sterility and validation methods, if applicable. See “Submission and Review of Sterility Information in Premarket Notification (510(k)) Submissions for Devices Labeled as Sterile.”
N	Shelf Life	Summary of methods used to establish that device performance is maintained for the entirety of the proposed shelf-life (e.g., mechanical properties, coating integrity, pH, osmolality), if applicable.
O	Biocompatibility	Information on the biocompatibility assessment of patient contacting materials, if applicable. See “Use of International Standard ISO 10993-1, ‘Biological evaluation of medical devices – Part 1: Evaluation and testing within a risk management process.’”
P	Software/Firmware	Submission of applicable software documentation, if applicable. See “Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices.”
Q	Cybersecurity/Interoperability	Submission of applicable information regarding the assessment of cybersecurity, if applicable. See “Content for Premarket Submissions for Management of Cybersecurity in Medical Devices” and “Design Considerations and Premarket Submission Recommendations for Interoperable Medical Devices.”
R	Electromagnetic Compatibility (EMC), Electrical, Mechanical, Wireless and Thermal Safety	Submission of the EMC, Electrical, Mechanical, Wireless and Thermal Safety testing for your device or summarize why testing is not needed. See “Electromagnetic Compatibility (EMC) of Medical Devices” and “Radio Frequency Wireless Technology in Medical Devices.”
S	Performance Testing	For non-in vitro diagnostic devices: Provide information on the non-clinical and clinical test reports submitted, referenced, or relied on in the 510(k) for a determination of substantial equivalence. See “Recommended Content and Format of NonClinical Bench Performance Testing Information in Premarket Submissions.” For in vitro diagnostic devices: Provide analytical performance, comparison studies, reference range/expected values, and clinical study information.
T	References	Inclusion of any literature references, if applicable.
U	Administrative Documentation	Inclusion of additional administrative forms applicable to the submission, including but not limited to a general summary of submission/executive summary (recommended), a Truthful and Accuracy Statement, and a 510(k) Summary or statement.
V	Amendment/Additional Information (AI) response	Inclusion of responses to Additional Information requests.

Important information in the eSTAR guidance

In Table 1 above, there are 14 hyperlinks to various FDA guidance documents. These links are extremely helpful when you have questions about a specific question. Unfortunately, the 510k electronic submission guidance document will quickly become out-of-date as guidance documents are updated and are made obsolete. In particular, one of the A-list final guidance documents planned for FY 2023 is the FDA cybersecurity guidance. We expect that guidance to be updated and released any day.

Posted in: 510(k), eSTAR

What is MDUFA V?

Posted by Alysha Bellarouse on October 10, 2022

MDUFA V is the agreement between the FDA and the medical device industry to fund the review of medical device submissions by the FDA.

What is MDUFA V?

The Medical Device User Fee and Modernization Act (MDUFMA or MDUFA) is a set of agreements between the Food and Drug Administration (FDA) and the medical device industry to provide funds for the Office of Device Evaluations (ODE) to review medical device submissions. FDA user fees were first authorized via MDUFMA in 2002 for FY 2003. Each MDUFA reauthorization has lasted five years, and FY 2023 will be the 21st year.

How are the MDUFA V user fees decided?

Section 738A(b)(1) of the FD&C Act requires that FDA consult with a range of stakeholders, including representatives from patient and consumer advocacy groups, healthcare professionals, and scientific and academic experts, in developing recommendations for the next MDUFA five-year cycle. The FDA initiated the reauthorization process by holding a public meeting on October 27, 2020, where stakeholders and other public members were allowed to present their views on the reauthorization. The following is a list of the four industry groups represented in the MDUFA V negotiations with the FDA:

The FD&C Act further requires that FDA continue meeting with the representatives of patient and consumer advocacy groups at least once every month during negotiations with the regulated industry to continue discussions of stakeholder views on the reauthorization and their suggestions for changes.

What are FDA user fees?

At the very core of it, the FDA user fees fund the FDA Office of Device Evaluation (ODE) budget. Without these user fees, the FDA cannot begin reviewing a medical device submission. This includes 510k, PMA, and De Novo submissions. Before the FDA assigns a reviewer to your submission, you must pay the appropriate device user fee in full unless eligible for a waiver or exemption. If you pay the user fee by credit card, you will need to allow a few extra days for the user fee to clear. Otherwise, your submission will be placed on “User Fee Hold.” Small businesses may qualify for a reduced fee. The FDA will announce the user fees for FY 2024 in a Federal Register notice next August 2023.

When does MDUFA V take effect?

Our team regularly checked the announcement of the MDUFA V user fees from August until last week’s announcement. The announcement of the FY 2023 user fees was delayed because Congress did not approve the MDUFA reauthorization until the last week of September. The new user fees were originally expected to take effect on October 1, 2022, but the announcement of actual user fees for 2022 was announced on October 5, 2022. This was two months later than expected.

Why was MDUFA V delayed, and will it happen again?

MDUFA V was delayed because the user fee reauthorization requires an act of Congress. The House of Representatives approved the Food and Drug Amendments of 2022 on June 8, 2022. However, the Senate did not file a bill until after the August recess. There were also differences between the legislation proposed by the House and the Senate. Therefore, to ensure that the FDA did not have to furlough employees when MDUFA IV funding expired, a temporary reauthorization was approved and signed by the President on September 30, 2022. The short-term continuing resolution is a temporary stopgap to continue funding the FDA until December 16, 2022. However, the continuing resolution covers funding for medical device user fees through September 30, 2027. Therefore, the device industry can expect the FDA to continue to operate regardless of the outcome of temporary policies that expire this December. Still, similar delays occurred with previous MDUFA reauthorization, and we expect more of the same US partisan politics between August 2027 and the November 2027 election.

How much did MDUFA V user fees increase?

The increase is dependent upon the fee type. Annual registration fees are increasing by 14.47% (i.e., $5,672 to $6,493). The MDUFA V user fees increased by a stupendous amount (+55.90%) from $12,745 to $19,870 for the 510k user fees. Yikes! De Novo Classification Requests increased by 17.79% from $112,457 to $132,464. Other submissions increased by similar amounts. For more details, check out the table below (also posted on our homepage).

20190810 075548 300x225 What is MDUFA V?

FDA User Fee FY 2023 represents a 55.90% increase in the 510(k) user fee

FY 2023 User Fees 1024x587 What is MDUFA V?

Do user fees ever decrease?

If we lived in a magical world where gas prices dropped and stayed low, the inflation-adjusted pricing would decrease for FDA user fees. That has happened once, but I fit into skinny jeans once too.

Why is August 1st important?

August 1st is the first day the FDA starts to accept Small Business Certification Requests for the new fiscal year. That means any small business that wants to keep small business status needs to reapply, and any new business that qualifies for small business status must also apply. The importance of applying for small business status is how much you could save on your submission. The FDA will complete its review of the Small Business Certification Request within 60 calendar days of receipt. Upon completion of the review by the FDA, the FDA will send you a decision letter with your small business designation number or a justification for denial.

Does small business status expire?

Yes, small business status expires. The small business status expires on September 30 of the fiscal year it is granted. A new MDUFA Small Business Certification Request must be submitted and approved each fiscal year to qualify as a small business. If you forget to reapply for small business status on August 1, you can reapply anytime during the year. Still, you will temporarily lose small business status from October 1 until the qualification is renewed. The good news is there is no fee associated with the submission of a Small Business Certification Request. For more information, please visit our webpage dedicated to small business qualifications.

Tags: FDA User Fees, MDUFA V

Posted in: 510(k), De Novo, FDA

FDA CCP now accepts FDA eSTAR & eCopy

Posted by Rob Packard on July 24, 2022

Finally, we can use the new FDA CCP to eliminate FedEx shipments, and 100% of your submissions will be electronic through the portal.

July 2022 Update for the FDA eCopy process

The FDA created a Customer Collaboration Portal (CCP) for medical device manufacturers. Originally, the portal’s purpose was to provide a place where submitters can track the status of their submissions and verify the deadlines for each stage of the submission review process. Last week, on July 19, the FDA emailed all active FDA CCP account holders that they can upload both FDA eCopy and FDA eSTAR files to the portal 100% electronically. Since our consulting team sends out submissions daily, everyone on the team was able to test the new process. If you have a CCP account, you no longer need to ship submissions via FedEx to the Document Control Center (DCC).

FDA CCP step-by-step uploading process

When you are uploading an FDA eCopy for medical device submission to the Document Control Center (DCC), using the new FDA CCP, the following steps are involved:

Confirm your eCopy complies with FDA’s eCopy guidance.
Compress your eCopy into a “.zip” file.
Sign in to the portal on the login page
Click on the “+” symbol on the left panel of the webpage (if you hover over the “+” symbol, you will see “Send a submission”)
Select your desired upload format (pre-submissions, meeting minutes, breakthrough device designations, and withdrawal letters must be submitted as an eCopy)
Click on the “Next” button that appears below the selection formats once a format is selected
Drag & drop your single “.zip” file here, or browse for it.
Click on “Send” button to complete the uploading process.
Verify that the FDA CCP site gives you a confirmation for the successful uploading of your submission.

FDA Q&A about the new FDA CCP Submission Uploading Process

Medical Device Academy Question: Who will be permitted to use the FDA CCP to upload submissions for the DCC? FDA Response: We will first offer this feature in batches to people like you who already use CCP so we can study its performance. We will then refine it and make it available to all premarket submitters.
Medical Device Academy Question: What do you need to use the FDA CCP? FDA Response: You don’t need to do anything to participate since you already use CCP. We will email you again when you can start sending your next submissions online.
Medical Device Academy Question: Suppose another consultant asks me to submit an eSTAR or eCopy for them, or I do this for a member of my consulting team. Is there any reason I cannot upload the submission using my account even though the other person is the official submission correspondent and their name is listed on the cover letter? FDA Response: The applicant and correspondent information of the submission is still used when logging the submission in. The submitter (i.e., the person uploading the submission) is not used in any part of the log-in process. The submission portal is essentially replacing snail mail only; once the DCC loads the submission, whether it be from a CD or an online source, the subsequent process is identical to what it used to be, for now.
Medical Device Academy Question: Is there any type of eCopy that would not be appropriate for this electronic submission process (e.g., withdrawal letters, MAF, or breakthrough device designations)? FDA Response: You can use the eCopy option to submit anything that goes to the DCC, so all your examples are fair game, though interactive review responses would still be emailed to the reviewer.
Medical Device Academy Question: How can I get help from the FDA? FDA Response: If you have questions, contact us at CCP@fda.hhs.gov.

Tags: Customer Collaboration Portal, eCopy, FDA CCP

Posted in: 510(k), De Novo

What’s new in the 2022 draft cybersecurity guidance?

Posted by Rob Packard on April 19, 2022

On April 8, 2022, the FDA released a new draft cybersecurity guidance document to replace the 2018 draft that the industry does not support.

Why was the draft cybersecurity guidance created?

Due to the ubiquitous nature of software and networked devices in the medical industry, the impact of cybersecurity attacks is becoming more frequent and more severe. The WannaCry Ransomeware Attack is just one example of this global cybersecurity issue. The FDA is responding to the need for stronger cybersecurity controls by issuing a new draft cybersecurity guidance for 2022.

The first four paragraphs of the introduction explain why we need this, and WannaCry is mentioned in the second paragraph of the background section. This new guidance is only a draft, but this is the FDA’s third attempt at regulating the cybersecurity of medical devices. The first guidance was finalized in 2014. That’s the 9-page guidance we currently have in effect. The guidance mentions risk 11 times and there is no mention of testing requirements or a bill of materials (BOM). The 2018 draft guidance (24-pages) met with resistance from the industry for a lot of reasons. One of the reasons mentioned by Suzanne Schwartz in an interview is the inclusion of a cybersecurity bill of materials (CBOM). The industry felt it would be too burdensome to disclose all of the hardware elements that are related to cybersecurity. Therefore, the FDA rewrote the 2018 draft and released a new draft on April 8, 2022 (49-pages).

Untitled presentation e1650071404761 What’s new in the 2022 draft cybersecurity guidance?

You might have expected the FDA to soften its requirements in the face of resistance from industry, but the new draft does not appear to be less robust. It is true that the CBOM was replaced by a software bill of materials (SBOM). However, the SBOM must be electronically readable and it must include:

the asset(s) where the software resides;
the software component name;
the software component version;
the software component manufacturer;
the software level of support provided through monitoring and maintenance from the software component manufacturer;
the software component’s end-of-support date; and
any known vulnerabilities.

You can be sure that the medical device industry will view providing an SBOM as a hefty burden. After all, a machine-readable SBOM is more complex than UDI labeling requirements. An SBOM will not fit on the “Splash Screen” for anyone’s software application. Companies may provide documentation through the company website with a link in their software to that information. The format of the information could be in the “Manufacturer Disclosure Statement for Medical Device Security (MDS2).” However, MDS2 is a 349-line item Excel spreadsheet to be used as a checklist (i.e. quite a bit longer than the GUDID data elements spreadsheet), and it took the FDA eight years to complete the transition for the UDI Final Rule (i.e. 2013 – 2021).

The 2018 draft cybersecurity guidance document from the FDA required a cybersecurity bill of materials (CBOM). CBOM was defined as “a list that includes but is not limited to commercial, open source, and off-the-shelf software and hardware components that are or could become susceptible to vulnerabilities.” Therefore, the FDA’s change from a CBOM to an SBOM eliminated the requirement to disclose the hardware components. Despite the change in disclosure requirements, manufacturers will still be expected to monitor potential hardware vulnerabilities to cybersecurity attacks. It should also be noted that the language in the PATCH Act (a new bill submitted to the House of Representatives and to the Senate for ensuring the cybersecurity of medical devices) specifically requires manufacturers “to furnish a software bill of materials as required under section 524B (relating to ensuring the cybersecurity).”

Structure of the draft cybersecurity guidance

The 2022 draft cybersecurity guidance organizes the requirements into four major principles:

cybersecurity as part of device safety and the quality system regulations
designing for security
transparency
submission documentation

The draft cybersecurity guidance recommends the implementation of a Secure Product Development Framework (SPDF). However, there is not much detail provided in the guidance for a SPDF. In the past, the term for this type of process was referred to as a Secure Software Development Lifecycle (i.e. Secure SDLC). However, in February 2022, the NIST Computer Security Resource Center (CSRC) released version 1.1 of the Secure SDLC guidance which is now titled “Secure Software Development Framework (SSDF) Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities.” This guidance provides guidance on the implementation of best practices for reducing the risk of software vulnerabilities because existing standards for managing the software development lifecycle do not explicitly address software security (e.g. IEC 62304-1:2015). The SSDF recommends implementing a core set of high-level secure software development practices that can be integrated into your SDLC process. Your software development team will also require cybersecurity training.

Design for security is the second principle of the draft cybersecurity guidance

Under this new draft cybersecurity guidance, the FDA will be evaluating the cybersecurity of devices based on the ability of the device to provide and implement the following security objectives:

Authenticity, which includes integrity;
Authorization;
Availability;
Confidentiality; and
Secure and timely updatability and patchability.

Transparency of cybersecurity information is for users

The draft cybersecurity guidance seeks to give device users more information pertaining to the device’s cybersecurity controls, potential risks, and other relevant information. This information will be in the form of an SBOM that is electronically readable. This information shall include disclosure of 1) known vulnerabilities or risks, 2) information to securely configure and update devices, and 3) communication interfaces and third-party software.

In addition to providing an SBOM, the FDA draft cybersecurity guidance includes requirements for cybersecurity labeling in section VI(A). There are 15 specific labeling requirements identified by the FDA for sharing with device users to improve the transparency of cybersecurity information. The first of these requirements is recommendations from the manufacturer for cybersecurity controls appropriate for the intended use environment (e.g., antimalware software, use of a firewall, password requirements). This first labeling requirement is identical to the 2018 draft guidance. Several of the other requirements are copied from the 2018 draft guidance, but others are new and/or reworded cybersecurity labeling requirements.

FDA Submission Documentation Requirements

The 2022 FDA draft cybersecurity guidance includes requirements for FDA submission documentation. Submission documentation must include a security risk management plan and report. The draft cybersecurity guidance explains on page 13 (numbered 9) that “performing security risk management is a distinct process from performing safety risk management as described in ISO 14971:2019.” Therefore, instead of using your safety risk management process, your software development team will need to have a different risk management process for software security. Details on the content for security risk management plans and reports can be found in AAMI TIR57:2016 – Principles for medical device security—Risk management. Appendix 2 also provides guidance for the inclusion of a) call flow diagrams, and b) information details for an architecture view.

Cybersecurity testing requirements for your FDA submission

The biggest impact of this new draft guidance may be the requirement for testing. The 2014 guidance has no testing requirement, the 2018 draft guidance mentioned testing 5 times in a few bullet points, but this new draft guidance mentions testing 43 times. The testing requirements for cybersecurity risk management verification include:

Security requirements
Threat mitigation
Vulnerability testing
Penetration testing

This guidance also includes a paragraph with multiple bullets of requirements for each of the four types of testing. This would essentially double the size and scope of the current software section for a 510k submission, and manufacturers will need to create new procedures and templates for their cybersecurity risk management process. For example, penetration testing requirements include the following elements:

Independence and technical expertise of testers,
Scope of testing,
Duration of testing,
Testing methods employed, and
Test results, findings, and observations.

Differences between the cybersecurity guidance documents

The following table provides a high-level overview comparing the four cybersecurity guidance documents released by the FDA, including the 2016 guidance on post-market management of cybersecurity:

Screenshot 2022 04 16 12.48.51 AM 1024x291 What’s new in the 2022 draft cybersecurity guidance?

Vulnerability management plans

The FDA draft cybersecurity guidance document also has a requirement for manufacturers to develop a plan for identifying and communicating vulnerabilities to device users after the release of the device. The FDA requires this plan to be included in your device submission. The vulnerability management plan should include the following information (in addition to the requirements of the 2016 guidance for postmarket cybersecurity management):

Personnel responsible;
Sources, methods, and frequency for monitoring for and identifying vulnerabilities (e.g. researchers, NIST NVD, third-party software manufacturers, etc.);
Periodic security testing to test identified vulnerability impact;
Timeline to develop and release patches;
Update processes;
Patching capability (i.e. rate at which update can be delivered to devices);
Description of their coordinated vulnerability disclosure process; and
Description of how manufacturer intends to communicate forthcoming remediations, patches, and updates to customers.

What’s the next step for the draft cybersecurity guidance?

In March the “Protecting and Transforming Cyber Health Care Act of 2022 (PATCH Act)” was introduced to the House of Representatives and the Senate. The goal of the PATCH Act is to enhance medical device security by requiring manufacturers to create a cybersecurity risk management plan for monitoring and addressing potential postmarket cybersecurity vulnerabilities. The FDA seeks comments on the draft cybersecurity guidance through July 7, 2022. Given the support of the new bill in the House of Representatives and Congress, it is likely that the FDA will get the support it needs for this new guidance.

Posted in: 510(k), Cybersecurity

Leave a Comment (0) →

Human factors process, can we make this easy to understand?

Posted by Rob Packard on April 12, 2022

90% of usability testing submitted to the FDA is unacceptable and the root cause is simply a failure to understand the human factors process.

If you submitted no usability testing to the FDA in your 510(k) submission, it would be obvious why the FDA reviewer identified usability as a major deficiency. However, you spent tens of thousands of dollars on usability testing that delayed the 510(k) submission by six months. Despite all of the time and money your company invested in the human factors process, it appears that you need to start over and repeat the entire process again. The CEO is furious, and he wants you to show him where in the 49-page FDA guidance it says that you have to do things differently.

Benefits from the human factors process

Use errors result in serious injuries and death
Easy to use products sell
You will prevent delays in regulatory approval

Why was your rationale for no usability testing rejected?

Unlike CE Marking technical files, the FDA does not require a usability engineering file for all products. Instead, the FDA determines if usability testing is required based upon a comparison of your device’s user interface and a competitor’s user interface (i.e. predicate device user interface). If the user interface is identical, then usability testing may not be required. Instead, your company should be able to write a rationale for not doing usability testing based upon equivalence with the predicate device. If there are differences in your user interface, you will need to provide use-related risk analysis (URRA), identify critical tasks, implement risk controls, and provide verification testing to demonstrate the effectiveness of the risk controls. Even if your device is “easier to use” or “simpler”, you still need to provide the documentation to support this claim in your submission. The FDA also does not allow comparative claims in your marketing for 510(k) cleared devices. Comparative claims require the support of clinical data.

What is the 10-step human factors process?

Define human factors for your device or IVD
Identify use errors
Conduct a URRA
Perform a critical task analysis
Conduct a risk control option analysis
Conduct formative usability testing
Implement risk controls
Conduct summative usability testing
Prepare HFE/UE documentation
Collect post-market surveillance data specific to use errors

There is a YouTube video describing these 10 steps at the bottom of this blog posting.

Why is formative testing needed?

Observational study to identify unforeseen use errors
Observational study to evaluate risk control options
What are the other types of studies?
Development of indications for use
Development of training materials

Why is the human factors process crazy expensive to outsource?

Human factors consultants need time to learn about your device
Consultants are more conservative because they cannot afford to fail
Justifying your choice of risk controls is difficult because you started too late
Your instructions for use (IFU) are inadequate
Consultants need to explain the human factors process to you
Recruiting subjects is marketing (which may not be their expertise)
You are paying for infrastructure (specialized testing facilities)
This is a team effort that requires many consulting hours collectively

Why was your Usability Engineering File refused?

Your company provided an application failure modes and effects analysis (aFMEA) to support your justification that residual risks are acceptable. The FDA guidance suggests using risk analysis tools such as an FMEA or fault-tree analysis, but deficiency letters from FDA reviewers recommend a use-related risk analysis (URRA) format that is totally different.

Example of a URRA Table provided by the FDA for the Human Factors Process

The primary problem with using an FMEA or Fault-Tree risk analysis tool is that these tools involve estimation of the severity of harm and the probability of occurrence of harm, while the FDA does not feel it is appropriate to estimate the probability of occurrence of harm. Instead, the FDA instructs companies to assume that use errors will occur and to implement risk controls to mitigate those risks (see URRA example above). Although “mitigation” is unlikely, and use risks will only be reduced, this is the approach the FDA wants companies to use. In addition, the FDA expects your company to provide traceability of risk control implementation to each use-related risk you identified and the FDA expects documentation of verification testing (i.e. usability testing) that shows your risk controls are effective. Finally, the FDA (and ISO 14971, Clause 10) expects you to collect and perform a trend analysis of use errors. Any use errors that are reported should be evaluated for the need to implement additional corrective actions to prevent future use errors. Blaming “user error” is not an acceptable approach.
You provided risk analysis and human factors testing in your 510(k) submission, but the FDA reviewer said you need to identify critical tasks and provide traceability to each critical task in your summative validation report. – Critical tasks are specifically mentioned in section 3.2 of the FDA guidance on applying human factors and usability engineering–and a total of 49 times throughout the guidance. However, “critical tasks” are not mentioned even once in ISO 14971:2019 or ISO/TR 24971:2020. The term “critical tasks” is not even found in IEC 62366-1:2015. There is mention of “tasks”, and “task” is a formal definition (i.e. Definition 3.14, “Task – one or more USER interactions with a MEDICAL DEVICE to achieve a desired result”). Therefore, companies that are familiar with the ISO Standards and CE Marking process frequently need training on the FDA requirements for the human factors process. After receiving training, then your company will be prepared to modify your usability engineering file documentation to comply with the FDA requirements for human factors.
You completed a summative validation protocol, but the FDA disagrees with your definition of user groups. – Each user has a different level of experience, training, and competency. Therefore, if you define the intended user population too broadly (e.g. healthcare practitioners), the FDA may not accept your summative usability testing. This is the reason that the human factors process begins with defining the human factors for your IVD or device. Radiologists, for example, have the following training pathway:
- graduate from medical school;
- complete an internship;
- pass state licensing exam;
- complete a residency in radiology;
- become board certified; and
- complete an optional fellowship.

Therefore, if you are developing imaging software, you need to make sure your user group includes radiologists that cover the entire range of competencies. In addition, most radiology images are taken by radiology technicians and then reviewed by the radiologist. Therefore, radiology technicians should be considered a completely different user group due to the differences in experience, training, and competency when compared to a radiologist. This simple example doubles the number of users needed because you have two user groups instead of one.

You evaluated 15 users, but the FDA reviewer is asking you to evaluate a larger number of users based upon a special controls guidance document. – The FDA guidance on human factors testing specifies a minimum of 15 users for each user group–not a minimum of 15 users. Therefore, for a device that is for Rx-only and OTC use, you will have at least two user groups that need to be evaluated independently. In addition, some devices have special controls guidance documents that specify usability testing requirements. For example, an OTC blood glucose meter must pass a 350-person lay-user study. Covid-19 self-tests are expected to pass a 30-person lay-user study as another example.
Your usability study was conducted in Australia, but the FDA insists that your usability study must be repeated in the USA. – Most people think of language being the primary difference between two countries, and therefore the author of a study protocol may not perceive any difference between the USA and Australia, Ireland, Canada, or the UK. However, this lack of ability to identify differences between cultural norms shows our own ignorance of cultural differences. International travelers learn quickly about the differences in the interface used for electrical outlets between the USA and other countries. There are also more subtle differences between cultures, such as in which direction do you toggle a light switch to turn on a light, up or down? For devices that are used in a hospital environment, it is critical to understand how your device will interact with other devices and how different hospital protocols might impact human factors.
The FDA reviewer indicated that your usability engineering file does not assess the ability of laypersons to self-select whether your OTC device is appropriate for them. – Devices and IVD devices may have contraindications or indications for use that are specific to an intended patient population or intended user population. In these cases, the user of the device or IVD needs to be able to “self-select” as included or excluded from use. The ability to self-select should be assessed as part of any OTC usability study. The ability to identify suitable and unsuitable patients for treatment is also a common criterion for a usability study involving prescription devices where a physician is the subject of the study.
The FDA reviewer indicated that you did not provide raw data collected by the study moderator. – Data collected during a human factors study is usually subjective in nature, and the FDA may want to conduct their own review and analysis of your data. Therefore, you cannot provide only a testing report that summarizes the results of your study. You must also provide the raw data for the study. It is permitted to provide the data in a tabular format that has been transcribed from paper case report forms or was recorded electronically. You should also consider scanning any paper forms for permanent retention or retaining the paper forms in case there is any question of accuracy in the transcription of the data collected. Finally, it is best practice to record videos of the study participants performing each task and answering interview questions. This will help in filling any gaps in the notes recorded by the moderator, and the recording provides additional objective evidence of the study results.
The FDA reviewer indicated that your study is not valid, because the training provided by moderators was not scripted and training decay was not considered in the design of the study. – Summative usability testing requires that users complete all of the critical tasks identified in your critical task analysis without assistance. It is permitted to provide training to the user prior to conducting the study if the device or IVD is for prescription use and healthcare practitioners are responsible for providing instruction to the user. However, any training provided must be scripted in advance and approved as part of the summative usability testing protocol. This ensures that every subject in the study receives consistent training. Unfortunately, the FDA may still not be satisfied with the design of your study if you do not allow sufficient time to pass between the time that training is provided to the user and when the subject uses the device or IVD for the first time. In general, one hour is the minimum amount of time that should pass between providing user training and when the device or IVD is used for the first time. This is referred to as “training decay” and the duration of time between your scripted training and the user performing critical tasks for the first time should be specified in your summative usability protocol. One solution to address both issues is to provide a video of the instructions to each subject 24-hours in advance of participation in the study.

Additional resources for the human factors process and usability testing

Tags: Design controls, Human Factors Process, IEC 62366-1, Usability

Posted in: 510(k), Design Control, Usability