Why modernize 21 CFR 820 to ISO 13485?

Leave a Comment / FDA, Human Factors, ISO 13485:2016, Post-Market Surveillance, Quality Management System, Software Verification and Validation / By Robert Packard

The FDA patches the regulations with guidance documents, but there is a desperate need to modernize 21 CFR 820 to ISO 13485.

FDA Proposed Amendment to 21 CFR 820

On February 23, 2022, the FDA published a proposed rule for medical device quality system regulation amendments. The FDA planned to implement amended regulations within 12 months, but the consensus of the device industry is that a transition of several years would be necessary. In the proposed rule, the FDA justifies the need for amended regulations based on the “redundancy of effort to comply with two substantially similar requirements,” creating inefficiencies. In public presentations, the FDA’s supporting arguments for the proposed quality system rule change rely heavily upon comparing similarities between 21 CFR 820 and ISO 13485. However, the comparison table provided is quite vague (see the table from page 2 of the FDA’s presentation reproduced below). The FDA also provided estimates of projected cost savings resulting from the proposed rule. What is completely absent from the discussion of the proposed rule is any mention of the need to modernize 21 CFR 820.

Are the requirements “substantively similar”?

The above table provided by the FDA claims that the requirements of 21 CFR 820 are substantively similar to the requirements of ISO 13485. However, there are some aspects of ISO 13485 that will modernize 21 CFR 820. The areas of impact are 1) software, 2) risk management, 3) human factors or usability engineering, and 4) post-market surveillance. The paragraphs below identify the applicable clauses of ISO 13485 where each of the four areas are covered.

Modernize 21 CFR 820 to include software and software security

Despite the limited proliferation of software in medical devices during the 1990s, 21 CFR 820 includes seven references to software. However, there are some Clauses of ISO 13485 that reference software that are not covered in the QSR. Modernizing 21 CFR 820 to reference ISO 13485 will incorporate these additional areas of applicability. Clause 4.1.6 includes a requirement for the validation of quality system software. Clause 7.6 includes a requirement for the validation of software used to manage calibrated devices used for monitoring and measurement. Clause 7.3 includes a requirement for validation of software embedded in devices, but that requirement was already included in 21 CFR 820.30. The FDA can modernize 21 CFR 820 further by defining Software as a Medical Device (SaMD), referencing IEC 62304 for management of the software development lifecycle, referencing IEC/TR 80002-1 for hazard analysis of software, referencing AAMI TIR57 for cybersecurity, and referencing ISO 27001 for network security. Currently, the FDA strategy is to implement guidance documents for cybersecurity and software validation requirements, but ISO 13485 only references IEC 62304. The only aspect of 21 CFR 820 that appears to be adequate with regard to software is the validation of software used for automation in 21 CFR 820.75. This requirement is similar to Clause 7.5.6 (i.e., validation of processes for production and service provisions).

Does 21 CFR 820 adequately cover risk management?

The FDA already recognizes ISO 14971:2019 as the standard for the risk management of medical devices. However, the risk is only mentioned once in 21 CFR 820. In order to modernize 21 CFR 820, it will be necessary for the FDA to identify how risk should be integrated throughout the quality system requirements. The FDA recently conducted two webinars related to the risk management of medical devices, but implementing a risk-based approach to quality systems is a struggle for companies that already have ISO 13485 certification. Therefore, a guidance document with examples of how to implement a risk-based approach to quality system implementation would be very helpful to the medical device industry.

Modernize 21 CFR 820 to include Human Factors and Usability Engineering

ISO 13485 references IEC 62366-1 as the applicable standard for usability engineering requirements, but there is no similar requirement found in 21 CFR 820. Therefore, human factors are an area where 21 CFR 820 needs to be modernized. The FDA has released guidance documents for the human factors content to be included in a 510k pre-market notification, but the guidance was released in 2016 and the guidance does not reflect the FDA’s current thoughts on human factors/usability engineering best practices. The FDA recently released a draft guidance for the format and content of human factors testing in a pre-market 510k submission, but that document is not a final guidance document and there is no mention of human factors, usability engineering, or even use errors in 21 CFR 820. Device manufacturers should be creating work instructions for use-related risk analysis (URRA) and fault-tree analysis to estimate the risks associated with use errors as identified in the draft guidance. These work instructions will also need to be linked with the design and development process and the post-market surveillance process.

Modernize 21 CFR 820 to include Post-Market Surveillance

ISO/TR 20416:2020 is a new standard specific to post-market surveillance, but it is not recognized by the FDA. There is also no section of 21 CFR 820 that includes a post-market surveillance requirement. The FDA QSR focuses on reactive elements such as:

21 CFR 820.100 – CAPA
21 CFR 820.198 – Complaint Handling
21 CFR 803 – Medical Device Reporting
21 CFR 820.200 – Servicing
21 CFR 820.250 – Statistical Techniques

The FDA does occasionally require 522 Post-Market Surveillance Studies for devices that demonstrate risks that require post-market safety studies. In addition, most Class 3 devices are required to conduct post-approval studies (PAS). For Class 3 devices, the FDA requires the submitter to provide a plan for a post-market study. Once the study plan is accepted by the FDA, the manufacturer must report on the progress of the study. Upon completion of the study, most manufacturers are not required to continue PMS.

How will the FDA enforce compliance with ISO 13485?

It is not clear how the FDA would enforce compliance with Clause 8.2.1 in ISO 13485 because there is no substantively equivalent requirement in the current 21 CFR 820 regulations. The QSR is 26 years old, and the regulation does not mention cybersecurity, human factors, or post-market surveillance. Risk is only mentioned once by the regulation, and software is only mentioned seven times. The FDA has “patched” the regulations through guidance documents, but there is a desperate need for new regulations that include critical elements. The transition of quality system requirements for the USA from 21 CFR 820 to ISO 13485:2016 will force regulators to establish policies for compliance with all of the quality system elements that are not in 21 CFR 820.

Companies that do not already have ISO 13485 certification should be proactive by 1) updating their quality system to comply with the ISO 13485 standard and 2) adopting the best practices outlined in the following related standards:

AAMI/TIR57:2016 – Principles For Medical Device Security – Risk Management
IEC 62366-1:2015 – Medical devices — Part 1: Application of usability engineering to medical devices
ISO/TR 20416:2020 – Medical devices — Post-market surveillance for manufacturers
ISO 14971:2019 – Medical Devices – Application Of Risk Management To Medical Devices
IEC 62304:2015 – Medical Device Software – Software Life Cycle Processes
ISO/TR 80002-1:2009 – Medical device software — Part 1: Guidance on the application of ISO 14971 to medical device software
ISO/TR 80002-2:2017 – Medical device software — Part 2: Validation of software for medical device quality systems

What is the potential impact of the US FDA requiring software, risk management, cybersecurity, human factors, and post-market surveillance as part of a medical device company’s quality system?

Why modernize 21 CFR 820 to ISO 13485? Read More »

Jan 4 2023

Best human factors questions?

1 Comment / 510(k), FDA, Human Factors, Usability / By Robert Packard / human factors

Best human factors questions to ask the FDA during a pre-submission meeting, and what information content do you need in a 510k?

Human factors questions to ask the FDA?

The FDA did not start enforcing the requirement to apply human factors and usability engineering to medical device design until 2017 because the final version of the human factors guidance document was not released until February 3, 2016. Approximately ninety percent of the human factors testing reports submitted to the FDA in 510k pre-market submissions are deficient because the 510k submission content only includes the final summative testing report. The FDA needs a complete usability engineering file, and the human factors information needs to comply with FDA guidelines for the format and content of a 510k pre-market submission–not just IEC 62366-1:2015.

What human factors information does the FDA want?

For several years, FDA submission deficiency letters indicated that you should not include the frequency of occurrence in your estimation of use-related risks. Still, the FDA never provided this information in a guidance document. On December 9, 2022, the FDA finally released a draft human factors guidance regarding the format and content of a 510k pre-market submission. The new draft guidance includes a use-related risk analysis (URRA) requirement in table 2 (copied below).

In this new draft FDA guidance, the FDA identifies three different human factors submission categories. For the first category, only a conclusion and high-level summary are needed. For the second category, a user specification is also needed. For the third category, you need a comprehensive human factors engineering report with the following elements described in Section IV of the draft FDA guidance:

Submission Category 1, 2, and 3

Conclusion and high-level summary

Submission Category 2 and 3

Descriptions of intended device users, uses, use environments and training
Description of the device-user interface
Summary of known use problems

Submission Category 3 only

Summary of preliminary analyses and evaluations
Use-related risk analysis to analyze hazards and risks associated with the use of the device
Identification and description of critical tasks
Details of validation testing of the final design

Before spending tens of thousands or hundreds of thousands of dollars on human factors testing, you want to ensure the FDA agrees with your human factors testing plan. Otherwise, you will pay for the testing twice: once for your initial submission and a second time in your response to the FDA request for additional information to address deficiencies. Testing can cost more than your electrical safety testing. The facility needs the right equipment and space for the testing; you need support personnel to set up the equipment; you need to recruit participants; you need to compensate participants; and you need device samples.

When can you ask the FDA human factors questions?

The FDA cannot provide consulting advice on a submission, and the agency will not review data during pre-submission meetings. The FDA can provide feedback on protocols, specifications, and scientific justifications. Therefore, you should submit questions to the FDA in a pre-submission when you have a draft protocol, a draft specification, or a draft justification for why a task is not critical. Pre-submissions are “non-binding.” You can change your design and approach to human factors. Therefore, don’t wait until your information is 100% finalized. Share your documentation at the draft stage during the development phase and before your design freeze. You need these answers when you are planning a study and obtaining quotes.

What are the best human factors questions to ask in a pre-sub?

In the FDA guidance for pre-submission meetings, the FDA provides suggested questions to ask:

Does the Agency have comments on our proposed human factors engineering process?
Is the attached use-related risk analysis plan adequate? Does the Agency agree that we have identified all the critical tasks?
Does the Agency agree with our proposed test participant recruitment plan for the human factors validation testing?

The above examples are only suggestions, but the best approach is to provide a brief example of what the human factors information will look like and ask the FDA to comment on the examples. The FDA does not have time to review data during a pre-sub meeting, but the FDA can review a few rows extracted from your URRA and comment on your proposed approach to the human factors process.

Human factors questions that are not appropriate

The FDA pre-submission guidance cautions you only to ask 3-4 questions for each meeting request because the FDA has difficulty answering more questions in a 60-minute teleconference. Therefore, you should not ask questions already answered in the FDA guidance. The new draft guidance includes examples of when a device modification can leverage existing human factors information and when new information is needed to support a premarket submission. Instead of asking a question specific to leveraging existing human factors information, provide your rationale for leveraging existing data and ask if the FDA has any concerns with your overall approach to human factors.

Recommended human factors action items

Create a procedure for your human factors process that includes detailed instructions for creating the information required in a usability engineering report and templates for each document.

Best human factors questions? Read More »

Oct 9 2019

Integrating usability testing into your design process

1 Comment / Design Control, Usability / By Robert Packard

This article explains how you should be integrating usability testing into your design control process–especially formative usability testing.

Why you should be integrating usability testing into the design

We recently recorded an updated usability webinar and released a usability procedure (SYS-048) with help from Research Collective–a firm specializing in human factors testing. After listening carefully to the webinar, and reading through the new usability procedure, I felt we needed to update our combined design/risk management plan to specify formative testing during phase 3 and summative (validation) testing during phase 4 of the design process. This is necessary to ensure your usability testing is interwoven with your risk management process. Integrating usability testing into all phases of your design process is critical–especially design planning (phase 1), feasibility (phase 2), and development (phase 3).

Integrating usability testing into your design plan helps identify issues earlier

During the usability training webinar, Research Collective provided a diagram showing the various steps in the usability engineering process. The first five steps should be included in Phases 1 and 2 of your design process. Phase 1 of the design process is planning. In that phase, you should identify all of the usability engineering tasks that need to be performed during the design process and estimate when each activity will be performed. The first of these usability activities is the identification of usability factors related to your device. Identifying usability factors is performed during Phase 2 of your design process before hazard identification.

Indentifying Usability Issues 300x209 Integrating usability testing into your design process

Before performing hazard identification, which should include identifying potential use errors, you need to identify five key usability elements associated with your device:

prospective device users during all stages of use must be defined
use environments must be identified
user interfaces must be identified
known use errors with similar devices and previous generations of your device must be researched
critical tasks must be described in detail and analyzed for potential use errors

Defining users must include the following characteristics: physical condition, education, literacy, dexterity, experience, etc. Use environment considerations may consist of low lighting, extreme temperatures or humidity, or excessive uncontrolled motion (e.g., ambulatory devices). User interfaces may include keyboards, knobs, buttons, switches, remote controllers, or even a touch screen display.

Often the best reason for developing a new device is to address an everyday use error that is inherent to the design of your current device model or a competitor’s product. Therefore, a thorough review of adverse event databases and literature searches for potential use errors is an important task to perform before hazard identification. This review of adverse event data and literature searches of clinical literature are key elements of performing post-market surveillance, and now ISO 13485:2016 requires that post-market surveillance shall be an input to your design process.

Finally, the step-by-step process of using your device should be analyzed carefully to identify each critical user task. User tasks are defined as “critical” for “a user task which, if performed incorrectly or not performed at all, would or could cause serious harm to the patient or user, where harm is defined to include compromised medical care.” Not every task is critical, all critical tasks must be identified, and ultimately you need to verify that each critical task is performed correctly during your summative (validation) usability testing.

Evaluating Risk Control Options – Formative Usability Testing in Phase 3 (Development)

Once your design team has conducted hazard identification and identified your design inputs (i.e., design phase 2), you will begin to evaluate risks and compare various risk control options. Risk control option analysis requires testing multiple prototype versions to assess which design has the optimum benefit/risk ratio. This is an iterative process that involves screening tests. For any use risks you identify, formative usability testing should be performed. Sometimes the risk controls you implement will create new use errors or new risks of other types. In this case, you must compare the risks before implementing a risk control with risks created by the risk control.

Ideally, each design iteration will reduce the risks further until all risks have been eliminated. The international risk management standard (ISO 14971) states that risks shall be reduced as low as reasonably practicable (ALARP). However, the European medical devices regulations require risks to be reduced as far as possible, considering the state-of-the-art. For example, all small-bore connectors in the USA are now required to have unique connectors that are incompatible with IV tubing Luer lock connections to prevent potential use errors. That requirement is considered “state-of-the-art.” If your device is marketed in both the USA and Europe, you will need to reduce errors as far as possible–before writing warnings and precautions in your instructions for use.

Reaching the point where use errors cannot be reduced any further may require many design iterations, and each iteration should be subsequently evaluated with formative usability testing. Formative testing can be performed with prototypes, rather than production equivalents, but the formative testing conditions should also address factors such as the use environment and users with different levels of education and/or experience. Ultimately, if the formative testing is done well, summative (validation) testing will be a formality.

Risk Control Effectiveness During Phase 4 – Summative Usability Testing during Verification

Once your team freezes the design, you will need to conduct verification testing. This includes integrating usability testing into the verification testing process. Summative (validation) testing must be performed once your design is “frozen.” If you are developing an electrical medical device, then you will need to provide evidence of usability testing as part of your documentation for submission to an electrical safety testing lab for IEC 60601-1 testing. There is a collateral standard for usability (i.e., IEC 60601-1-6). For software as a medical device (SaMD), you will also be expected to conduct usability testing to demonstrate that the user interface does not create any user errors.

When you conduct summative (validation) testing, it is critical to make sure that you are using samples that are production equivalents rather than prototypes. Also, it is crucial to have your instructions for use (IFU) finalized. Any residual risks for use errors should be identified in the precautions section of your IFU, and the use of video is encouraged as a training aid to ensure use errors are identified, and the user understands any potential harm. When the summative testing is performed, there should be no deviations and no use errors. Inadequate identification of usability factors during Phase 2, or inadequate formative testing during Phase 3, is usually the root cause of failed summative testing. If your team prepared sufficiently in Phase 2 and 3, the Phase 4 results would be unsurprisingly successful.

Additional Training Resources for Usability Engineering

The following additional training resources for usability engineering may be helpful to you:

Integrating usability testing into your design process Read More »