Software security, what is the best time to test cybersecurity?

The 2022 draft cybersecurity guidance from the FDA emphasizes the need to design devices to be secure and the need to design devices capable of reducing emerging cybersecurity risks throughout the total product lifecycle. Designing devices for security must be built into your original design plan, or you will need to modify your device for improved security just to obtain initial 510(k) clearance from the FDA. What is not clear from the guidance or standards is when you need to conduct security testing or repeat tests.

Planning Cybersecurity Tests

As with all quality system processes, cybersecurity testing should begin with a plan. There are two models typically used for the design and development process: Waterfall Diagram (typical of hardware development) and V-Diagram (typical of software development).

V-Diagram

How are design plans for SaMD different from other design plans?

Most of the verification testing for software as a medical device (SaMD) is 1) conducted virtually, 2) tests software code in a “sandbox,” and 3) involves internally developed testing protocols. In contrast, verification testing for other types of devices involves 1) physical devices, 2) testing at a 3rd party lab, and 3) involves international standards and testing methods. The biggest differences between SaMD verification testing and other device verification testing are the speed and cost of the testing. SaMD verification is much faster and less expensive. Therefore, if your software design documentation is efficient, you can complete more design iterations. This is why software developers use the V-diagram to model the design and development process instead of the “waterfall” diagram.

Where do the requirements to test cybersecurity belong in your design plan?

A design plan documents the design and development process for your device. You must establish, maintain, and update the plan as the project progresses. There is no required format, but auditors and the FDA will audit your Design History File (DHF) for compliance with your plan. You are required to document the following content in your plan:

• Stages of development
• Reviews at each design and development stage
• Verification, validation, and design transfer activities at each stage
• Responsibilities and authorities for the design project
• Methods you are using to ensure traceability of user needs, software hazards, software requirements, software design specifications, and software testing reports
• Human resources needed for your design project, including competency

Software Design Inputs

In the early stages of the software development lifecycle, you must select an appropriate threat model and perform a hazard analysis for software security. These security hazards need to be included as design inputs in your software requirements specification (SRS). The need for updateability and patchability should also be included as design inputs. 

In parallel with your SRS, you will need to create a User Specification. The SRS and User Specification will determine the use cases and call-flow views that require verification testing later in your software development process. After the SRS has been approved, you will need to create a software design specification (SDS). Each item in the SDS should be traceable to an item in the SRS. The SDS items that trace to security hazards are your risk controls. Each risk control will require you to test cybersecurity to verify risk control effectiveness. At this point, you will need to create your testing protocols for security.

System Testing Protocols to Test Cybersecurity

Testing protocols should include a boundary analysis and rationale for boundary assumptions. Testing protocols should also include vulnerability testing. The FDA recommends the following vulnerability testing:

1. Abuse cases, malformed, and unexpected inputs,
   1. Robustness
   2. Fuzz testing
2. Attack surface analysis,
3. Vulnerability chaining,
4. Closed box testing of known vulnerability scanning,
5. Software composition analysis of binary executable files, and
6. Static and dynamic code analysis, including testing for credentials that are “hardcoded,” default, easily guessed, and easily compromised.

Does your development budget include security testing? 

Design control training traditionally emphasizes the importance of “freezing” design outputs before starting verification testing to prevent the need for repeating any of the verification testing. The reason for this is that verification testing is expensive, and it is time-consuming to produce additional verification samples. In contrast, SaMD is guaranteed to be changed multiple times during the verification testing process as software bugs are identified. Therefore, software developers focus on the velocity of developing code and testing that code. One exception to this is penetration testing. Penetration testing is usually conducted once your code is final because it is more expensive than other software verification and validation testing and it would need to be repeated each time the software is updated or patched.

Penetration Testing

Penetration testing is another method used to test cybersecurity that would probably be conducted in parallel with simulated use testing to validate performance and the effectiveness of human factors risk controls. Penetration testing could be at the system level in a sandbox environment, or it can be performed on a sample device in a simulated use environment. Your penetration testing documentation should include the following:

1. independence and technical expertise
2. scope of testing
3. duration of testing
4. testing employed, and
5. test results, findings, and observations

Postmarket cybersecurity management

For CE Marked products, there is a requirement for a postmarket surveillance plan (i.e., PMS plan) to be submitted as part of your technical file. The US FDA does not currently have this requirement for Class 1 and Class 2 devices, but Class 3 devices (i.e., PMA) and devices with humanitarian device exemptions (HDE)  are required to submit a PMS plan as part of the premarket submission. The US FDA also requires a postmarket cybersecurity management plan to be submitted for premarket submissions of Class 2 and Class 3 devices. You should create your postmarket cybersecurity management plan during your verification and validation activities, and the final version should be approved at the time of product release.

If you need additional resources or training related to cybersecurity, you may be interested in the following:

• Cybersecurity Work Instruction (WI-007)
• FDA Cybersecurity Training Video (YouTube)
• Cybersecurity Labeling Requirements Video (YouTube)