Blog

Archive for Software Verification and Validation

Software Service Provider Qualification and Management

What is your company’s approach to qualifying a software service provider and managing software-as-a-service (SaaS) for cybersecurity?

The need for qualifying and managing your software service provider

Most of the productivity gains of the past decade are related to the integrartion of software tools into our business processes. In the past, software licenses were a small part of corporate budgets and the most critical software tools helped to manage material requirements planning (MRP) functions and customer relationship management (CRM). Today, there are software applications to automate every business process. Failure of a single software service provider, also known as “Software-as-a-Service” or (Saas), can paralyze your entire business. In the past, business continuity plans focused on labor, power, inventory, records, and logistics. Today our business continuity plans also need to expand for the inclusion of software service providers, internet bandwidth, websites, email, and cybersecurity. This new paradigm is not specific to the medical device industry. The medical device industry has become more dependent upon its supply chain due to the ubiquity of outsourcing, and what happens to other industries will eventually filter its way into this little collective niche we share. With that in mind, how do we qualify and manage a software service provider?

Threats to software service providers (Kaseya Case Study)

Two years ago the WannaCry ransomeware attack affected 200,000 computers, 150 countries, and more than 80 hospitals.

Wana Decrypt0r screenshot Software Service Provider Qualification and Management

Kaseya isn’t a hospital. Kaseya is a software service provider company. So why is this example relevant to the medical device industry?

The ransomware attack on Kaseya was severe enough that both CISA and the FBI got involved, and it compromised some Managed Service Providers (MSPs) and downstream customers. This supply chain ransomeware attack even has its own Wikipedia page. The attack prompted Kaseya to shut down servers temporarily. None of this is a critique of Kaseya or their actions. They were merely the latest high-profile victim of a cyberattack in the news. Now cybercriminals are attacking your supply chain. We want to emphasize the concepts and considerations of this type of attack as it pertains to your business.

What supplier controls do you require for a software service provider?

If you are a manufacturer selling a medical device under the jurisdiction of the U.S. FDA, you need to comply with 21 CFR 820.50 (i.e. purchasing controls). The FDA requires an established and maintained procedure to control how you are ensuring what your company buys meets the specified requirements of what you need. Many device manufacturers only consider suppliers that are making physical components, but a software service provider may be critical to your device if your device includes software or interacts with a software accessory. A software service provider may also be involved with quality system software, clinical data management, or your medical device files. Do you purchase software-as-a-service or rely upon an MSP for cloud storage?

You need to determine if your software service provider is involved in document review or approval, controlling qulaty records, Protected Health Information (PHI), or electronic signature requirements. You don’t need a supplier quality agreement for all of the off-the-shelf items your company purchases. For example, it would be silly to have Sharpie sign a supplier quality agreement because you occasionally purchase a package of highlighters. On the other hand, if you are relying upon Docusign to manage 100% of your signed quality records, you need to know when Docusign updates its software or has a security breach. You should also be validating Docusign as a software tool, and there should be a back-up of your information.

21 CFR 820.50 requires that you document supplier evaluations to meet specified and quality requirements per your “established and maintained” procedure. The specified requirements for this supplier might include the following:

  • How much data storage do you need?
  • How many user accounts do you need?
  • Do you need to unique electronic IDs for each user?
  • Do you need tech support for the software service?
  • Is the software accessed with an internet browser, or is the software application-based, or both?
  • How much does this software service cost?
  • Is the license a one time purchase? Or is it a subscription?

The quality requirements for a supplier like this may look more like these questions;

  • How is my information backed up?
  • Can I restore previous file revisions in the case of corruption?
  • How can I control access to my information?
  • Can I sign electronic documents? If yes, is it 21 CFR Part 11 compliant?
  • Does this supplier have downstream access to my information? (can the supplier’s suppliers see my stuff?)
  • Do I manage PHI? If so, can this system be made HIPAA compliant? What about HITECH?
  • What cybersecurity practices does this supplier utilize?
  • How are routine patches and updates communicated to me?

A risk-based approach to supplier quality management

ISO 13485:2016 requires that you apply a risk-based approach to all processes, including supplier quality management. A risk-based approach should be applied to suppliers providing both goods and services. For example, you may order shipping boxes and contract sterilization services. Both companies are suppliers, but in this example the services provided by the contract sterilizer is associated with a much higher risk than the shipping box supplier. Therefore, it makes sense that you would need to exercise greater control over the sterilizer. A software service providers are much like a contract sterilizer. SaaS is not tangible but the service provided may have a high level of risk and potential impact on your quality management system. Therefore, you need to determine the risk associated with SaaS before you can evaluate, control, and monitor a software service supplier.

First you need to document the qualification of a new supplier. It would be nice if your cloud service provider had a valid ISO 13485:2016 certification. You would then have an objectively demonstratable record of their process controls and know that they are routinely audited to maintain that certification. They would also understand and expect to undergo 2nd party supplier audits because they operate in the medical device industry. Alternatively, a software service provider may have an ISO 9001:2015 certification. This is a  general quality system certification that may be applied to all products or services. In the absence of quality system certification, you can audit a potential supplier. For some suppliers, this makes sense. However, many companies that are outside of the medical device industry do not even have a quality system because it is not required or typical of their industry. For the ones that do, though, you can likely leverage their existing certifications and accreditations.

Cybersecurity standards you should know

Most cloud service providers will not have ISO 13485 certification, because it is a quality management standard specific to the medical device industry. However, you might look for some combination of the following ISO standards that may be relevant to a software service provider:

  • ISO/IEC 27001 Information Technology – Security Techniques – Information Security Management Systems – Requirements
  • ISO/IEC 27002:2013 Information Technology. Security Techniques. Code Of Practice For Information Security Controls
  • ISO/IEC 27017:2015 Information Technology. Security Techniques. Code Of Practice For Information Security Controls Based On ISO/IEC 27002 For Cloud Services
  • ISO/IEC 27018:2019 Information Technology – Security Techniques – Code Of Practice For Protection Of Personally Identifiable Information (PII) In Public Clouds Acting As PII Processors
  • ISO 22301:2019 Security And Resilience – Business Continuity Management Systems – Requirements
  • ISO/IEC 27701:2019 Security Techniques. Extension to ISO/IEC 27001 and ISO/IEC 27002 For Privacy Information Management. Requirements And Guidelines

Does your software service provider have SOC reports?

%name Software Service Provider Qualification and Management

The acronym “SOC” stands for Service Organization Control, and these reports were established by the American Institute of Certified Public Accountants. SOC reports are internal controls that an organization utilizes and each report is for a specific subject. SOC reports apply to varying degrees for SaaS and MSP Suppliers

The SOC 1 Report focuses on Internal Controls over Financial Reporting. Depending on what information you need to store on the cloud, this report could be more applicable to the continuity of your overall business than specifically to your quality management system.

The SOC 2 Report addresses what level of controls an organization places on the five Trust Service Criteria: 1) Security, 2) Availability, 3) Processing Integrity, 4) Confidentiality, and 5) Privacy. As a medical device manufacturer, these areas would touch on control of documents, control of records, and process validation, among other areas of your quality system. Some suppliers may not share a SOC 2 report with you, because of the amount of confidential detail provided in the report.

The SOC 3 Report will contain much of the same information that the SOC 2 Report contains. They both address the five Trust Service Criteria. The difference is the intended audiences of the reports. The SOC 3 is a general use report expected to be shared with others or publicly available. Therefore, it doesn’t go into the same intimate level of detail as the SOC 2 report. Specifically, information regarding what controls a system utilizes is very brief if identified at all compared to the description and itemized list of controls in the SOC 2 Report.

Others ways to qualify and manage your software service provider

SOC reports will help paint a picture of the organization you are trying to qualify. You will also need to evaluate the supplier on an on-going basis. It is essential to know if the supplier is subject to routine audits and inspections to maintain applicable certifications and accreditations. For example, if their ISO certificate lasts for three years, you should know that you should follow up with your supplier for their new certificate at least every three years. On the other hand, if they lose certification, it may signify that the supplier can’t meet your needs any longer and you should find a new supplier.

There is a long list of standards, certifications, accreditations, attestations, and registries that you can use to help qualify a SaaS or MSP supplier. One such registry is maintained by Cloud Security Alliance (i.e. the CSA STAR registry). “STAR” is an acronym standing for Security, Trust, Assurance, and Risk. CSA describes the STAR registry in their own words as:

“STAR encompasses the key principles of transparency, rigorous auditing, and harmonization of standards outlined in the Cloud Controls Matrix (CCM) and CAIQ. Publishing to the registry allows organizations to show current and potential customers their security and compliance posture, including the regulations, standards, and frameworks they adhere to. It ultimately reduces complexity and helps alleviate the need to fill out multiple customer questionnaires.”

Some of the questions your supplier qualification process should be asking about your SaaS and MSP suppliers include:

  • Why do I need this software service?
  • Which standards, regulations, or process controls need to be met?
  • What is required for qualifying a suppliers providing SaaS or an MSP?
  • How will you monitor a software service provider?

ISO certification, SOC reports, and the CSA STAR registry are supplier evaluation tools you can use supplier qualification and monitoring. When you use these tools, make sure that you ask open ended questions instead of close-ended questions. Our webinar on supplier qualification provides several examples of how to convert your “antique” yes/no questions into value-added questions.

Are your suppliers qualified Supplier Evaluation Tools Software Service Provider Qualification and Management

Your software service provider should be able to provide records and metrics demonstrating effectiveness of their cybersecurity plans. Below are three examples of other types of records you might request:

  • Cloud Computing Compliance Controls Catalogue or “C5 Attestation Report”
  • System Security Plan for Controlled Unclassified Information in accordance with NIST publication SP 800-171
  • Privacy Sheild Certification to EU-U.S. Privacy Shield or Swiss-U.S. Privacy Shield

The priacy shield certification may be especially important for companies with CE Marked devices in order to comply with the European Union’s General Data Protection Regulation (GDPR) or Regulation 2016/679.

A final consideration for supplier qualification is, “Who are the upstream suppliers?” It is essential to know if you new supplier or their suppliers will have access to Protected Health Information (PHI). Since you have less control of your supplier’s subcontractors, you may need to evaluate how your supplier manages their supply chain and which general cybersecurity practices your supplier’s subcontractors adhere to.

Additional cybersecurity, software validation, and supplier quality resources

For more resources on cybersecurity, software validation, and supplier quality management please check out the following resources:

Learn how to quickly perfect your 510k cybersecurity documentation rvp 8 12 2021 Software Service Provider Qualification and Management

Posted in: Cybersecurity, Software Verification and Validation, Supplier Quality Management

Leave a Comment (0) →

Cybersecurity FDA Guidance for Devices with Software and Firmware

This article reviews the FDA guidance for premarket and post-market cybersecurity of medical devices with software and firmware—including requirements for reporting of field corrections and removals.

Cybersecurity with custom aspect ratio Cybersecurity FDA Guidance for Devices with Software and Firmware

Hospitals, home health systems, and medical devices are more connected now than ever. The automatic communication between medical devices and network systems is improving the efficiency and accuracy in the world of healthcare. Medical devices are capable of more computing, analysis, reporting, and automation to improve the speed and quality of patient care. Along with technological advances, new risks and concerns are also introduced. The risk of hackers exploiting vulnerabilities in networks and software is inevitable. The FDA introduced guidance for both pre-market and post-market cybersecurity to assist manufacturers in developing effective controls to protect patients and users. Cybersecurity protection requires Identification, Protection, Detection, Response, and Recovery.

The first step is incorporating processes and procedures to improve device cybersecurity into your quality management system. You should have a specific cybersecurity plan to outline the steps necessary to ensure a safe and secure medical device.

Identify Cybersecurity Risks

The key to understanding and assessing the cybersecurity risks involved with your device begin in the early stages of design development. At the start of the risk management process, you need to identify the essential safety and performance requirements of the device. You need to identify any potential cybersecurity vulnerabilities that could impact safety or performance, as well as the specific harms that could result if the vulnerability was exploited. In assessing the specific vulnerabilities, the FDA recommends using the Common Vulnerability Scoring System (CVSS). There is a CVSS calculator available online through NIST. The overall score is calculated based on different factors such as attack vector (local, adjacent network, network), access complexity (high, medium, low), authentication (multiple, single, none), the impact of confidentiality (none, partial, complete), exploitability (unproven that exploit exists, proof of concept code, functional exploit exists), remediation level (official fix, temporary fix, workaround, unavailable), collateral damage potential (low, medium, high), etc. This score is used in the hazard analysis in determining the level of risk.

Cybersecurity Protection

The process of assessing the exploitability and harms can also assist in determining mitigations that can be implemented to reduce the cybersecurity risk. During the design process, the FDA expects you to implement as many protections as practicable. Protections include:

  • Limit Access to Trusted Users
    • Password protection strengthened password requirements
    • User authentication
    • Layered privileges based on user role
  • Limit Access to Tampering
    • Physical locks on devices and/or communication ports
    • Automatic timed methods to terminate sessions
  • Ensure Trusted Content
    • Restrict software or firmware updates to authenticated code
    • Systematic procedures for authorized users to download software and firmware only from the manufacturer
    • Ensure capability of secure data transfer, use of encryption

Cybersecurity Detection

The FDA also requires you to implement features that allow for security compromises to be detected, recognized, logged, timed, and acted upon during regular use. You should develop and provide information to the end-user concerning appropriate actions to take upon the detection of a cybersecurity event. Methods for retention and recovery should be provided to allow recovery of device configuration by an authenticated privileged user.

If you include off-the-shelf (OTS) software in your device, you are responsible for the performance of the software as part of the device. All software changes to address cybersecurity vulnerabilities of the OTS software need to be validated. You need to maintain a formal business relationship with the OTS vendor to ensure timely notification of any information concerning quality problems or corrective actions. Sometimes you will need to involve the OTS vendor to correct cybersecurity vulnerabilities.

Post-Market Surveillance

Once you complete the hazard analysis, mitigation implementation, validations, and has deployed their device for use – your activities shift to post-market management. Several QMS tools can assist in the cybersecurity processes post-market, including complaint handling, quality audits, corrective and preventive action, ongoing risk analysis, and servicing. A critical component of every cybersecurity program is the monitoring of cybersecurity information sources to assist in the identification and detection of risk. You should maintain contact with third-party software suppliers for the identification of new vulnerabilities, updates, and patches that come available.

There are many sources that companies should follow for information relating to cybersecurity, including independent security researchers, in-house testing, software or hardware suppliers, healthcare facilities, and Information Sharing and Analysis Organizations (ISAO). Involvement in ISAOs is strongly recommended by the FDA and reduces your reporting burden if an upgrade or patch is required post-market. ISAOs share vulnerabilities and threats that impact medical devices with their members. They share and disseminate cybersecurity information and intelligence pertaining to vulnerabilities and threats spanning many technology sectors, and are seen as an integral part of your post-market cybersecurity surveillance program.

Response and Recovery

If you identify a cybersecurity vulnerability, there are remediation and reporting steps that need to occur. Remediation may involve a software update, bug fixes, patches, “defense-in-depth” strategies to remove malware or covering an access port to reduce the vulnerability. Uncontrolled risks should be remediated as soon as possible and must be reported to the FDA according to 21 CFR 806. Certain circumstances remove the reporting requirement. The decision flowchart below can be used to determine the reporting requirements.

Cybersecurity software change decision tree Cybersecurity FDA Guidance for Devices with Software and Firmware

In addition to reporting corrections and removals, the FDA identifies specific content to be included in PMA periodic reports regarding vulnerabilities and risks. If you have a Class III device, you should review that section thoroughly to ensure annual report compliance.

If a device contains software or firmware, cybersecurity will be an important component of the risk management processes, and continual cybersecurity management will be necessary to ensure the ongoing safety and effectiveness of your device. If you need more help with cybersecurity risk management of your medical device, please schedule a free 15-minute call with Medical Device Academy by clicking on the link below.

Click here to schedule a 15 minute call 300x62 Cybersecurity FDA Guidance for Devices with Software and Firmware

Posted in: Software Verification and Validation

Leave a Comment (0) →

Define medical device software verification and validation (V&V)

This article defines software verification and validation (V&V) for medical devices. The article also provides an overview of the CE Marking application and 510k submission requirements for medical devices containing software. Finally, we provide a link to our free download of a webinar on 510k software documentation.

Software Validation and Verification 1 Define medical device software verification and validation (V&V)

Software Verification and validation is an essential tool for ensuring medical device software is safe. Software is not a piece of metal that can be put into a strain gauge to see if the code is strong enough not to break. That’s because software is intangible. You can’t see if it is in the process of failing until it fails. The FDA is concerned about software safety since many medical devices now include software. Software failure can result in serious injury or even death to a patient. This places significant liability on the device manufacturer to ensure their software is safe. One way to ensure software safety is to perform software verification and validation (V&V).

What is software verification and validation (V&V)?

Definitions of software verification and validation confuse most people. Which tasks are software verification? And which tasks are software validation? Sometimes the terms are used interchangeably. Even the FDA does not clearly define the meaning of these two terms for software. For example, in the FDA’s design control guidance document the following definitions are used:

“Verification means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled.”

“Validation means confirmation by examination and provision of objective evidence that the particular requirements for specific intended use can be consistently fulfilled.”

Specific intended use requirement…specified requirements…what is the difference? To understand the difference between the two terms, the key is understanding “Intended Use.” It is asking the question: “What is the software’s intended use?”

“Intended Use” is not just about a bunch of engineers sitting around a table coming up with really fresh ideas. “Intended Use” refers specifically to the patient/customer of the software and how it fulfills their needs (i.e., “User Needs”). Systematic identification of user needs is required, and the software must address the user needs. Identification of user needs is done through customer focus groups, rigorous usability studies, and consultation with subject matter experts such as doctors and clinicians providing expert insight.

“Intended Use” also ensures the safety of the process through the process of “Hazard Analysis,” whereby any hazard that could potentially cause harm to the patient/customer is identified. For each identified hazard, software requirements, software design, and other risk controls are used to make sure the hazard does not result in harm, or if it does, the severity of the harm is reduced as far as possible.

So if “Validation” ensures user needs are met, what is “Verification” and how does it apply to the software development process. “Verification” ensures that the software is built correctly based on the software requirements (i.e., design inputs), with regard to each task the software must perform (i.e., unit testing), during communication between software modules (i.e., integration testing) and within the overall system architecture (i.e., system-level testing). This is accomplished by rigorous and thorough software testing using prospectively approved software verification protocols.

CE Marking requirements for software verification and validation (V&V)

European CE Marking applications include the submission of a technical file that summarizes the technical documentation for the medical device. To be approved for CE Marking by a Notified Body, the device must meet the essential requirements defined in the applicable EU directive. The technical file must also include performance testing of the medical device in accordance with the “State of the Art.” For software, IEC/EN 62304:2006, medical device software – software life cycle processes, is considered “State of the Art” for the development and maintenance of software for medical devices. This standard applies to stand-alone software and embedded software alike. The standard also identifies specific areas of concern, such as software of unknown pedigree (SOUP). As with most medical device standards, the standard provides a risk-based approach for evaluation of SOUP acceptability and defines testing requirements for SOUP.

FDA requirements for software verification and validation (V&V)

For 510k submissions to the US FDA, section 16 of the 510k submission describes the software verification and validation (V&V) activities that have been conducted to ensure the software is safe and effective. There are 11 documents that are typically included in this section of the submission for software with a moderate level of concern:

  1. Level of Concern
  2. Software Description
  3. Device Hazard Analysis
  4. Software Requirement Specification (SRS)
  5. Architecture Design Chart
  6. Software Design Specification (SDS)
  7. Traceability Analysis
  8. Software Development Environment Description
  9. Verification and Validation Documentation
  10. Revision Level History
  11. Unresolved Anomalies (Bugs or Defects)

The FDA does not require compliance with IEC 62304 as the European Regulations do, but IEC 62304 is a recognized standard, and manufacturers must comply with all applicable parts of IEC 62304 if they claim to follow IEC 62304. The FDA also provides a guidance document for the general principles of software validation.

Additional Resources

If you are interested in learning more about the documentation requirements for a 510k submission of a software medical device, please click here to download a free recording of our 510k software documentation webinar.

Medical Device Academy also has a new live webinar scheduled for Tuesday, January 5, 2016, @ Noon (EST). The topic is “Planning Your 2016 Annual Audit Schedule. We are also offering this live webinar as a bundle with our auditor toolkit.

About the Author

Nancy Knettell is the newest member of the Medical Device Academy’s consulting team, and this is her first blog contribution to our website. Nancy is an IEC 62304 subject matter expert. To learn more about Nancy, please click here. If you have suggestions for future blogs or webinars on the topic of medical device software, please submit your requests to our updated suggestion box.

Posted in: Software Verification and Validation

Leave a Comment (0) →