Author name: Robert Packard

Auditing Risk Management Files

3 Comments / Auditing, CE Marking, ISO 14971:2019 (Risk Management), ISO Auditing, Remote Auditing / By / ,

What do you look at and look for when you are auditing risk management files to ISO 14971 and the new Regulation (EU) 2017/745?

Your cart is empty

Next week, November 15th @ Noon EST, you will have the opportunity to watch a live webinar teaching you what to look at and what to look for when you are auditing risk management files to Regulation (EU) 2017/745 and ISO 14971. Risk Management Files are one of the essential requirements of technical documentation required for CE Marking of medical devices. Most quality system auditors are trained on how to audit to ISO 13485:2016 (or an earlier version of that standard), but very few quality system auditors have the training necessary to audit risk management files.

Why you are not qualified to audit risk management files

Being a qualified lead auditor is not enough to audit the risk management process. When you are auditing a risk management file, you need risk management training and lead auditor training. To audit the risk management process, you will also need training on applicable guidance documents (i.e., ISO/TR 24971:2020) and applicable regulations (i.e., Regulation 2017/745 and/or Regulation 2017/746). There may also be device-specific guidance documents that specify known risks and risk controls that are considered state-of-the-art.

Creating an audit agenda

Once you have scheduled an audit of risk management files, and assigned a lead auditor, then the lead auditor needs to create an audit agenda. The audit can be a desktop audit that is performed remotely, or it can be an on-site audit. Regardless of the approach, the audit should include interviewing participants in the risk management process documented in the risk management file. As a rule of thumb, I expect a minimum of 30 minutes to be spent interviewing the process owner and one or more other participants. Then I spend an additional 60 minutes of auditing time reviewing documents and records.

Your audit agenda should specify the following items at a minimum:

  1. the method of auditing to be used,
  2. date(s) of the audit,
  3. the duration of the audit,
  4. the location of the audit, and
  5. the auditing criteria.

The auditor(s) and the auditee participants should be identified in the audit agenda. Finally, you should specify which documents and records are required for audit preparation. These documents will be used to help identify audit checklist questions and to determine a sampling plan for the audit. At a minimum, you will need a copy of the risk management procedure and a list of the risk management files that are available to audit. You may also want to request the audit plan for each of those risk management files.

What did you look at and look for during your risk management audit?

When you audit the risk management process, you could take any of the following approaches or a combination of more than one. You could audit the process according to the risk management procedure. You could audit the process according to the risk management plan(s) for each risk management file. You could audit using the process approach to auditing. Finally, you could audit in accordance with specific requirements in the ISO 14971:2019 standard and applicable regulations (i.e., Regulation 2017/745). Regardless of which approach you take, your audit notes and the audit report should identify which documents and records you sampled and what you looked for in each document. Providing only a list of the documents is not enough detail.

Creating an auditing checklist for risk management files

Auditors with limited experience are taught to create an audit checklist by creating a table that includes each of the requirements of the audit criteria. For a risk management file, this would include a list of each of the requirements in ISO 14971 for a risk management file (i.e., Clause 9???). However, this approach is more like the approach that you should be using for a gap analysis. The better approach for creating an audit checklist for risk management files is to start by creating a turtle diagram. In the “process inputs” section (i.e., step 2 of 7), you would add questions derived from your review of the risk management plan(s). In the “process outputs” section (i.e., step 3 of 7), you would add questions specific to the risk management report and other records required in a risk management file. In the “with whom” section (i.e., step 5 of 7), you would add questions related to training and competency. You might also identify additional people involved in the risk management process, other than the process owner, to interview as a follow-up trail. In the “how done” section (i.e., step 6 of 7), you would add questions specific to the procedure and forms used for the risk management process. Finally, in the “metrics” section (i.e., 7 of 7), you would verify that the company is conducting risk management reviews and updating risk management documentation in accordance with the risk management procedure and individual risk management plan(s).

Audits are just samples

Just because you can generate a lot of questions for an audit checklist does not mean that you are required to address every question. Audits are intended to be a “spot check” to verify the effectiveness of a process. You should allocate your auditing resources based on the importance of a process and the results of previous audits. I recommend approximately three days for a full quality system audit, and approximately 90-minutes should be devoted to a process unless it is the design control process (i.e., Clause 7.3 of ISO 13485) which typically requires three to four hours due to the importance and complexity of the design controls process. Therefore, you should schedule approximately 30 minutes to interview people for the risk management process and approximately 60 minutes should be reserved for reviewing documents and records. With this limited amount of time, you will not be able to review every record or interview everyone that was involved in the risk management process. This is why auditors always remind auditees that an audit is just a sampling.

Which records are required in a risk management file?

The contents of a risk management file is specified in ISO 14971:2019, Clause 4.5. There are only four bullets in that section, but the preceding sentence says, “In addition to the requirements of other clauses of this document.” Therefore, your risk management file should address all of the requirements in ISO 14971:2019. What I recommend is a virtual risk management folder for each risk management file. As the auditor, you should also request a copy of the risk management policy and procedure. An example of what this would look like is provided below. The numbers in front of each subfolder correspond to the sub-clause or clause for that requirement in ISO 14971:2019.

Risk Management File Example Auditing Risk Management FilesWhich records are most valuable when auditing risk management files?

As an auditor, I typically focus on three types of targets when auditing any process. First, I will sample any corrective actions implemented in response to previous audit findings. Second, will sample documents and records associated with any changes made to the process. Changes would also include any changes that were made to individual risk management files or the creation of a new risk management file. Finally, my third target for audit sampling is any item that I feel is at risk for safety or performance failures. The severity of the safety or performance failure is also considered when prioritizing audit sampling. In the context of a risk management file, I always verify that production and post-production activities are being conducted as planned. I try to verify that risk analysis documentation was reviewed for the need to update the documentation in response to complaints and adverse events.

More auditor training on risk management files

We are recording a live webinar intended to teach internal auditors and consultants how to perform a thorough audit of risk management files against the requirements of the new European Regulation (EU) 2017/745 and ISO 14971.

PXL 20221101 183748328 Auditing Risk Management Files
Auditing Risk Management Files
In this new webinar, you will learn how to conduct a process audit of risk management files. You will learn what to look at and what to look for in order to verify compliance with Regulation (EU) 2017/745 and ISO 14971:2019. The webinar will be approximately one hour in duration. Attendees will be invited to participate in the live webinar and receive a copy of the native slide deck. Anyone purchasing after the live event will receive a link to download the recording of the live event and the native slide deck.
Price: $64.50

In addition to this webinar on auditing risk management files, we also have other risk management training webinars available. The webinar on auditing risk management files will be hosted live on November 15, 2022 @ Noon EST (incorrect in the live video announcement).

Auditing Risk Management Files Read More »

What is the De Novo review timeline?

4 Comments / De Novo / By

The new FDA goal is to reduce the De Novo review timeline to 150 days for 70% of De Novo submissions, but how long does it take now?

What is an FDA De Novo submission?

An FDA De Novo submission is an application submitted to the FDA for creating a new device product classification. There are three classifications of devices by the FDA: Class 1, Class 2, and Class 3. Class 1 devices are the lowest-risk devices, and they only require general controls. Class 2 devices are moderate-risk devices that require “Special Controls,” and Class 3 are high-risk devices that require Pre-Market Approval (i.e., PMA). De Novo applications can only be submitted for Class 1 and Class 2 devices, and most of the De Novo submissions require clinical data to demonstrate that the clinical benefits of the new device classification outweigh the risks of the device to patients and users. It’s the need for clinical data that is partly responsible for the longer De Novo review timeline.

What is the De Novo review timeline?

Initially, the FDA required that Class 2 devices must be first submitted as a 510k submission. If the device did not meet the criteria for a 510k, then the company could re-submit a De Novo Classification Request to the FDA. On July 9, 2012, the regulations were revised to allow companies to submit De Novo Classification Requests directly. This makes sense because some devices have novel indications for use, and submission of a 510k would be a complete waste of time in money. For example, the first SARS-COV-2 test had to be submitted as a De Novo by Biofire to obtain permanent approval for the test instead of emergency use authorization (EUA). This change in 2012 dramatically reduced the De Novo review timeline.

On October 4, 2021, the FDA published a final rule for De Novo Classification Requests. This new regulation identified the De Novo review timeline as 120 calendar days. Even though 120 days is 30 days longer than the FDA review clock for a 510k, the actual timeline to review De Novo submissions was much longer.

Every five years, when Congress reauthorizes user fee funding of the FDA, new MDUFA goals are established. The draft MDUFA performance goals (which impact FDA funding) were published recently. The specific performance goal to review De Novo submissions is:

FDA will issue a MDUFA decision within 150 FDA Days for 70% of De Novo requests.

There are two problems with this goal. First, the term “FDA Days” is based on calendar days minus the number of days the submission was placed on hold, and we don’t have any visibility into the number of days submissions are placed on hold. In the past, submissions could be placed on hold multiple times during the Refusal to Accept (RTA) screening process, and the “FDA Days” is reset to zero days each time the company receives an RTA hold letter. In addition, even after the submission is finally accepted, the FDA places the submission on hold when they request additional information (i.e., AI Hold). RTA and AI Hold periods can last up to 180 days, and during the Covid-19 pandemic, companies were allowed to extend this up to 360 days.

The second problem with the MDUFA goal is that we only have visibility into the outcome of De Novo submissions that were granted. More than 60 De Novo submissions are submitted each year, but the number of De Novo Classification Requests granted ranged between 21 and 30 over the past three years. Therefore, the 50%+ of De Novo applications denied could skew the % of submissions that meet the MDUFA goal for the De Novo review timeline.

What is the FDA track record in reviewing a De Novo?

Every CEO I speak with asks the same question: “How long does the FDA review take?” In preparation for a webinar I taught about De Novo Classification Requests in 2019, I researched the latest De Novo review timelines. I expected the review timelines to be close to 150 calendar days because the FDA decision goal was 150 FDA days. The 150-day goal was set in 2018 when Congress approved MDUFA IV. The 2019 data held two surprises:

  1. only 21 De Novo requests were granted in 2019, and
  2. the average review timeline was 307 calendar days (i.e., the range was 108 days to 619 days).

FDA days are not the same as calendar days. Only 23.8% of De Novo submissions were reviewed within 150 calendar days. The FDA doesn’t calculate the number of FDA days as calendar days, but there is no way to know how much time each De Novo spent on hold publicly. Upon seeing the announcement of a new decision goal for MDUFA V on October 5, 2022, I decided to revisit my previous analysis.

De Novo review timeline What is the De Novo review timeline?

*Only 9+ months of data for 2022, because data was collected on October 17, 2022.

We can blame the Covid-19 pandemic for the slower De Novo review timeline during the past few years, but you would expect a longer average duration in 2020 if that was the root cause of the FDA’s failure to achieve the MDUFA IV target of 150 calendar days. You would also expect 2021 to have the longest review timelines. Instead, the review timelines are the slowest for 2022. The number of De Novo submissions remains small, and therefore it is hard to be conclusive regarding the root cause of the failure to reach the 150-day decision goal. In addition, the percentage of De Novo applications granted within 150 calendar days was lowest in 2021, as you would expect if the reason for delays is primarily due to the Covid-19 pandemic.

Is there any good news?

The FDA is allowing the new eSTAR templates to be used for De Novo Classification Requests. These new electronic submission templates standardize the format of all 510k and De Novo submissions for FDA reviewers. The eSTAR also forces companies to answer all questions in the FDA reviewer’s checklist to ensure the submission is complete and accurate before the new submission is submitted to the FDA.

The new eSTAR templates were first used in 2021, and our firm has observed shorter overall review timelines and fewer deficiencies identified by FDA reviewers when they submit an “Additional Information Hold” (AI Hold) to companies.

How can the FDA improve De Novo timelines?

The FDA, industry, and Congress seem to be taking the same approach pursued five years ago to improve the review timeline for De Novo submission. MDUFA V authorized additional user fees for De Novo submissions (i.e., 17.8% increase), and the FDA will be authorized to hire additional employees each year during MDUFA V if the performance goals are met. However, there are three other options that the FDA and industry should have seriously considered during the FDA-industry negotiations.

The first option that should have been considered is to allow third-party reviewers to review the elements of a De Novo that are identical to a 510k submission:

  1. sterilization validation
  2. shelf-life testing
  3. biocompatibility testing
  4. software validation
  5. electrical safety testing
  6. EMC testing
  7. wireless testing
  8. interoperability testing
  9. benchtop performance testing
  10. animal performance testing
  11. human factors engineering

The above approach would require blended pricing where the FDA charges a smaller user fee than a Standard De Novo user fee, and the third-party reviewer charges a smaller fee than a 510k. The combined cost would be higher than the FDA Review of a De Novo, but this would reduce the number of hours the FDA needs to complete their review of a De Novo, and it would allow for pricing that is much lower than the De Novo standard user fee for qualified small businesses.

A second approach would be to pilot a modular review approach. A modular review would be similar to modular reviews for PMA submissions. In a modular review, the FDA can review most submission sections and provide feedback before the human clinical performance data is available. This would not help the few De Novo submissions that do not include human clinical performance data, but this would have a profound positive impact on most De Novo projects. First, the FDA would be able to complete the review of all sections in the submission except the human clinical performance data without delaying the final De Novo decision. Second, a successful review of non-clinical data by the FDA would give investors more confidence to fund pivotal clinical studies required to complete the De Novo submission.

A third approach would be for the FDA to force manufacturers to submit testing plans and protocols as pre-submissions to the FDA. This approach would give the FDA more familiarity with each device and the testing plan before reviewing the data. This approach would also reduce the hours FDA reviewers spend reviewing data that doesn’t meet the requirements and writing deficiencies. This approach would also give investors more confidence to fund De Novo projects for all V&V testing.

What is the De Novo review timeline? Read More »

FDA CCP = e-Submission of FDA eSTAR + eCopy + SBD requests

6 Comments / 510(k), De Novo / By / , ,

The FDA CCP keeps getting better. Now, you can request small business determination (SBD) online through the FDA CDRH portal.

Information Needed to Create Small Business Determination SBD Request 1024x621 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

Preparing your FDA eSubmission for the FDA CCP

There are three possible formats for an FDA eSubmission to the FDA CCP for CDRH: 1) FDA eCopy, 2) FDA eSTAR, and 3) Small Business Determination (SBD) Request. We explain each in sequence below:

FDA eCopy:

The FDA eCopy is no longer an acceptable format for a 510k submission. You must use the FDA eSTAR template for a 510k submission. In contrast, pre-submission meeting minutes and withdrawal letters require an FDA eCopy. An FDA eCopy is also required for Sprints (i.e., pre-sub variation for Breakthrough devices and STeP devices). For a De Novo Submission and 513(g) submissions, you have the option of using the FDA eSTAR or the FDA eCopy (BUT use the FDA eSTAR or PreSTAR respectively). Here are the preparation steps:

  1. Confirm your eCopy complies with FDA’s eCopy guidance.
  2. Compress your eCopy into a “.zip” file.

FDA eSTAR (also includes FDA PreSTAR):

Confirm your eSTAR is complete. There are three ways to know that your eSTAR is complete and ready for upload, but don’t worry too much. The FDA validated that the FDA CCP will automatically detect an incomplete eSTAR and will not allow it to be uploaded.

  • Bold Font at the top of the eSTAR or PreSTAR template is Green

eSTAR is complete 1024x148 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

  • All of the color-coded bars on the left side of the template are Green or Grey

eSTAR color coded key 1024x337 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

  • All of the sections of the template in the verification section are found on the left side

Verification section complete 1024x301 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

Small Business Determination Request:

The ability to upload your small business qualification documents as an eSubmission instead of sending a hardcopy via courier is a new feature enabled by the FDA on Saturday night (i.e., September 29, 2024). Confirm your eCopy complies with FDA’s eCopy guidance.

FDA CCP step-by-step uploading process

When you are uploading a submission to the FDA CCP (i.e., CDRH Customer Collaboration Portal), you must perform the following steps:

#1 Sign in to the portal on the login page. If you don’t already have your own account, you can Sign up in less than 5 minutes.

FDA CCP login 536x1024 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

#2 Accept the FDA User Terms and Conditions

User Terms and Conditions 1024x533 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

#3 You have three options one you accept the FDA user terms and conditions. One option is to check on the status of prior eSubmissions on the Home page. Your second option is to send a new FDA eCopy or FDA eSTAR. For this second option you click on the “+” symbol on the left panel of the webpage (if you hover over the “+” symbol, you will see “Send a submission”) or you can expand the left panel as I have done in the picture below. The third option (i.e., the newest FY2025 option) is to create a request for small business determination (i.e., SBD).

FDA CCP options FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

#4 If you selected “Send a submission” you can upload your FDA eCopy or FDA eSTAR. The top of the page has the following instructions “Send your submission before 16:00 ET on a business day for us to process it the same day. Once you send a submission through the portal, you do not need to physically ship any copies to FDA. The online process replaced the physical process.”

FDA CCP send a submission page 1024x436 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

#5 Click on the “Next” button that appears below the selection formats once a format is selected.

#6 Drag & drop your single “.zip” file here, or browse for it.

#7 The webpage will ask you for a short description of your submission. The company name and type of submission is sufficient [e.g., “Medical Device Academy – presubmission meeting minutes (i.e., Q24xxxx.A001)”]. Click on “Send” button to complete the uploading process.

Send Step 1024x528 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

#8 Verify that the FDA CCP site gives you a confirmation for the successful uploading of your submission. I always create a screen capture of the confirmation page once I hit send, but that is unnecessary but faster to insert in an email to your boss. You will receive an email seconds later.

Confirmation that eCopy was sent 1024x556 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

The process is identical for the FDA eSTAR, but you DO NOT need to zip an FDA eSTAR or PreSTAR.

Starting a Small Business Determination (SBD) Request

The third type of submission for the FDA CCP is a small business determination. This new option was enabled last Saturday night (i.e., September 28, 2024)….what a thoughtful birthday present from the FDA to me. Just click on the document icon beneath the plus sign. The website will take you to the “Create a request” page where you can begin your SBD request. Before you start this process, you will need to prepare some documents:
#1 Tax return for the most recent tax year
#2 Organization ID number (watch our YouTube Video)

#3 Other information on FDA Form 3602 (see our Small Business Qualification webpage)
FDA CCP Create a Small Business Determination SBD Request 1024x247 FDA CCP = e Submission of FDA eSTAR + eCopy + SBD requests

July 2022 Update for the FDA eCopy process (FDA CCP is launched!)

Finally, we can use the new FDA CCP to eliminate FedEx shipments, and 100% of your submissions will be electronic through the portal. The FDA created a Customer Collaboration Portal (CCP) for medical device manufacturers. Initially, the portal’s purpose was to provide a place where submitters could track the status of their submissions and verify the deadlines for each stage of the submission review process. On July 19, 2022, the FDA emailed all active FDA CCP account holders that they can upload both FDA eCopy and FDA eSTAR files to the portal 100% electronically. The FDA released a draft eSTAR guidance as well. Since our consulting team sends out submissions daily, everyone on the team was able to test the new process. If you have a CCP account, you no longer need to ship submissions via FedEx to the Document Control Center (DCC).

FDA Q&A about the new FDA CCP Submission Uploading Process

  1. Medical Device Academy Question: Who will be permitted to use the FDA CCP to upload submissions for the DCC? FDA Response: We will first offer this feature in batches to people like you who already use CCP so we can study its performance. We will then refine it and make it available to all premarket submitters.
  2. Medical Device Academy Question: What do you need to use the FDA CCP? FDA Response: You don’t need to do anything to participate since you already use CCP. We will email you again when you can start sending your next submissions online.
  3. Medical Device Academy Question: Suppose another consultant asks me to submit an eSTAR or eCopy for them, or I do this for a member of my consulting team. Is there any reason I cannot upload the submission using my account even though the other person is the official submission correspondent and their name is listed on the cover letter? FDA Response: The applicant and correspondent information of the submission is still used when logging the submission in. The submitter (i.e., the person uploading the submission) is not used in any part of the log-in process. The submission portal is essentially replacing snail mail only; once the DCC loads the submission, whether it be from a CD or an online source, the subsequent process is identical to what it used to be, for now.
  4. Medical Device Academy Question: Is there any type of eCopy that would not be appropriate for this electronic submission process (e.g., withdrawal letters, MAF, or breakthrough device designations)? FDA Response: You can use the eCopy option to submit anything that goes to the DCC, so all your examples are fair game, though interactive review responses would still be emailed to the reviewer.
  5. Medical Device Academy Question: How can I get help from the FDA? FDA Response: If you have questions, contact us at CCP@fda.hhs.gov.

FDA CCP = e-Submission of FDA eSTAR + eCopy + SBD requests Read More »

Risk management policy – Do you have one?

1 Comment / ISO 14971:2019 (Risk Management) / By

ISO 14971:2019 includes a requirement for top management to define and document a risk management policy, but do you have one?

Screen capture for POL 005 1024x542 Risk management policy Do you have one?

Your risk management procedure is not your risk management policy

ISO 14971:2019 includes a requirement for a risk management policy and a risk management procedure. The word procedure is defined (Clause 3.13), a “specified way to carry out an activity or a process,” but there is no definition for policy. Both of these words begin with the letter “p,” but they are not the same.  There is no guidance for a risk management policy in either of the European device regulations for CE Marking and there is no guidance in the US FDA’s regulations. In fact, there is not even a specific cause of the international risk management standard that is specific to the requirement for a risk management policy. The word “policy” only appears in ISO 14971 seven times, but the last occurrence provides the best explanation:

  • Appendix A2.4.2 states that “because [ISO 14971] does not define acceptable risk levels, top management is required to establish a policy on how acceptable risks will be determined.”

If someone responsible for risk management activities does not understand this distinction, this shows that risk management training may not be adequate.

Can you have a different policy for each product family?

The purpose of the policy is to establish how the acceptability of risks will be determined. However, not all devices have the same benefit-risk ratio. Therefore, if you have product families with high and low risks, then you should address this in your policy with specific criteria for each device family or create a separate risk management policy for each product family. For example, if your company is focused on designing and developing products for diabetics, you will not have the same benefit-risk profile for a Class 2 glucose reader and lancet for Type 2 diabetics that you have for an automated Class 3 insulin pumps for Type 1 diabetics. In general, separate criteria within one policy are preferred over separate policies to reduce the number of documents that must be managed.

Is there a required format for a risk management policy?

The ISO 14971:2019 standard does not include a specific format or content requirement for your risk management policy. Instead, information about the format and content of a risk management policy is provided in Annex C of ISO/TR 24971:2020. This is a guidance document, and therefore you can choose an alternate approach if you provide a justification for its equivalence. If you choose the approach recommended in Annex C, the following elements should be included:

  • purpose;
  • scope;
  • factors and considerations for determining acceptable risk;
  • approaches to risk control;
  • requirements for approval and review.

You can download Medical Device Academy’s template for a risk management policy (POL-005) by completing the form below.

What are the factors for determining acceptable risk?

There are four possible factors to consider when determining your risk management policy:
  1. Applicable regulatory requirements;
  2. Relevant international standards;
  3. State-of-the-Art;
  4. And stakeholder concerns.

An example of regulatory requirements being applied to the determination of acceptable risks is the special controls defined in 21 CFR 880.5730 for insulin pumps. The special controls requirements outlined by the FDA specify design inputs as well as verification and validation requirements. The requirements are also organized into systems that comprise an insulin pump. For the digital interface requirements, the regulation specifies:

  • secure pairing to external devices;
  • secure data communication between the pump and connected devices;
  • sharing of state information between devices;
  • ensuring the pump continues to operate safely when receiving data that is outside of the boundary limits that are specified as inputs;
  • a detailed process and procedure for sharing pump interface specifications with connected devices.

The hazard implied by the fourth requirement above is that the pump will stop without warning or deliver the incorrect amount of insulin if the data from a continuous glucose sensor is outside of the input specifications. This design input is then addressed by a software design specification established by your company. To verify that your software risk controls are adequate, you will need to execute a verification protocol that automatically inputs a series of values that are outside of the boundary limits specified. Every time a change is made to the software, these boundary limits will need to be re-verified as part of your automated regression analysis to make sure software changes did not have an unintended effect on the device.

For software and use-related hazards, you will not be able to estimate the probability of occurrence of harm. Therefore, you shall assess the acceptability of risks based upon the severity of harm alone. Risk acceptability criteria shall be recorded in your risk management plan and the criteria shall align with your risk management policy. Ideally, these criteria are based upon international standards. For the example of an interoperable insulin pump, the following international standards are applicable:

  • ISO 14971, application of risk management to medical devices
  • IEC 62366-1, application of usability engineering to medical devices
  • IEC 62304, medical device software – software lifecycle processes

For the state-of-the-art, there are three examples provided in the ISO/TR 24971 guidance for how to this relates to your risk management policy:

  1. “Leakage currents of the medical device are state of the art, demonstrated by compliance to the limits and tests regarding leakage current of IEC 60601-1.
  2. Dose accuracy of the delivery device are state of the art, as demonstrated by compliance to the limits and tests regarding dose accuracy of ISO 11608-1.
  3. Protection against mechanical failure caused by impact is on the same level as or better than a similar medical device, as demonstrated by comparative test such as drop test.”

Stake holder concerns is the fourth factor to consider when creating your risk management policy. Stakeholder concerns may be identified in clinical literature. However, the current trend is an emphasis on patient-reported outcome (PRO) data and post-market surveillance. Post-market surveillance is a requirement in ISO 13485, Clause 8.2.1. However, the new European MDR and IVDR have new requirements for post-market surveillance data in the technical documentation. Health Canada updated the medical device regulations to include post-market surveillance summary reports, and even the FDA is trying to develop methods for using real-world data and real-world evidence to make regulatory decisions.

Approaches to risk acceptability

The European device regulations require that a benefit/risk analysis be conducted for all risks and the overall residual risk of your device. The EU regulations also do not permit risk acceptability to consider economic impact. The EU regulations also require that risks are reduced as far as possible. Therefore, if your company is seeking CE Marking, there is only one acceptable approach suggested in ISO/TR 24971, Annex C.2: “reducing risk as far as possible without adversely affecting the benefit-risk ratio.” This is also the approach specified in our risk management procedure (SYS-010).

Requirements for review and approval of the risk policy

Requirements for approval and review of the risk management policy should be specified in the policy itself. This should specify who needs to approve that the policy is acceptable and how often the policy needs to be reviewed. Section 4.2.2 of ISO 14971 also requires that top management review the risk management process for its effectiveness. In general, we recommend that this review of the risk management process be incorporated into the management review process. Therefore, we also believe that this would be the ideal time to review the risk management policy. Generally, this is more frequently than is typically required, but if your risk management process is being reviewed for effectiveness then you have all of the necessary inputs available to review the policy as well.

Risk management policy – Do you have one? Read More »

What is a CAPA Board? and Do you need one?

1 Comment / CAPA / By

A CAPA Board is a team responsible for making sure that all CAPAs are completed on time and the actions taken are effective.

Many of the medical device companies we work with have to open a CAPA for their CAPA process because they fail to implement all the actions that were planned, they fail to implement corrective actions as scheduled, or the actions implemented fail to be effective. When we investigate any process, we typically see one of five common root causes:

  1. top management is not committed to the CAPA process (we can’t fix this)
  2. procedures and/or forms are inadequate
  3. people responsible do not have sufficient training
  4. management oversight of the process is neglected
  5. there are not enough resources to do the work

Creating a CAPA Board can address four of these potential root causes, but the CAPA Board needs to understand how to work effectively.

Creating a CAPA Board shows a commitment to quality

Sometimes top management only pays lip service to quality. Top management’s actions demonstrate that quality is a cost-center, and they do not view quality as contributing to the revenue of the company. Instead, quality is viewed as a “necessary evil” like death and taxes. If this describes your company, sharpen your resume and find a new job. Quality is essential to selling medical devices and quality is the responsibility of everyone in the company. The Management Representative is responsible for “ensuring promotion and awareness” (see Clause 5.5.2c of ISO 13485) of regulatory and quality system requirements. This person should be training others on how to implement best practices in quality system management. One person or one department should never be expected to do most of the work related to the quality system.

A CAPA Board should be a cross-functional team of managers that help each other maintain an effective CAPA process. This means: 1) corrections are completed on time, 2) corrective and preventive actions are completed on time, and 3) each CAPA is effective. In order to do this consistently, the CAPA Board needs to work together as a team on the CAPA process. The CAPA Board doesn’t look for someone to blame. Instead, the CAPA Board rotates their responsibilities regularly, everyone is cross-trained on the roles within the CAPA Board, and the team passes tasks from one person or department that is overloaded to another person or department that has the resources to complete the tasks effectively and on time. A professional team must anticipate holes in task coverage, and someone on the team needs to communicate to the rest of the team which hole they are addressing. You can’t wait until the coverage gap is obvious and then have everyone jump into action. If you do this, your effectiveness will resemble a soccer team of 9-year-olds. 

Is your CAPA procedure the root cause?

In most companies, the problem is not the CAPA procedure. Clauses 8.5.2 and 8.5.3 of ISO 13485 are quite specific about each step of the CAPA process, and therefore it is easy to write a procedure that includes all of the required elements. The CAPA procedure is also one of the first procedures that auditors and inspectors review, and therefore any deficiencies in your procedure are usually addressed after one or two audits. If you feel that your CAPA procedure needs improvement, the above link explains how to write a better CAPA procedure. You might also consider asking everyone that is responsible for the CAPA process to provide suggestions on how to improve your procedure to streamline the process and clarify the instructions. The best approach is to have a small group (i.e. 3 to 5 people) of middle-level managers, from different departments, assigned to a CAPA Board with the responsibility of improving the CAPA process and procedure. If you have a large company, you might consider rotating people through the CAPA Board each quarter instead of having a larger group.

Does your CAPA Board have sufficient training?

Everyone can benefit from more training–even instructors will periodically engage in refresher training. Before someone is assigned to work on a CAPA, that person needs to be trained. Nobody should be assigned to a CAPA Board unless they are prepared to become an expert in the CAPA process. Some companies will only require people to sign a training record that states they read and understood the CAPA procedure. However, you must also demonstrate that your training was effective and the person is competent at the task assigned. Therefore, we recommend training people on CAPAs by training them with a CAPA training webinar and evaluating the effectiveness of the training by having each person complete a quiz. The use of a training webinar will ensure that each employee receives the same training, and the quiz will provide objective evidence that they understood the training (i.e. it was effective). If you have a CAPA Board, each person on the board should be involved in your CAPA training, and it is their responsibility to make sure people in their department have been trained effectively.

Competency is the hardest thing to demonstrate for any task. You can do this by verifying that the person has performed this task in one or more prior jobs (e.g. resume). If the person does not have evidence of working on CAPAs in their previous employment, then you will need someone that is already competent in the CAPA process to observe each person completing CAPAs and providing feedback. Once each person has demonstrated successful completion of multiple CAPAs, then the expert can attest to their competency in a training record with references to each of the successful CAPAs that were completed. If you are the person assigning a CAPA or individual tasks to people, do not assign the role of investigation, or writing the CAPA, to anyone that has not already demonstrated competency unless you are assessing them for competency. Everyone on the CAPA Board should either already be competent in the CAPA process or another expert on the CAPA Board should be in the process of training them to become a CAPA expert.

Average CAPA Aging Graph What is a CAPA Board? and Do you need one?

CAPA Boards are responsible for management oversight of the CAPA process

The most common method for management oversight of the CAPA process is to discuss the status of CAPAs at a Management Review. This information can be presented by the Management Representative, but assigning the presentation of CAPA status to another person on your CAPA Board will delegate some of the Management Review tasks and gives other people practice at presenting to a group. Some companies only conduct a Management Review once per year, but this makes it impossible to review CAPAs that were initiated immediately after a Management Review unless the CAPA takes more than a year to implement. Even if your company conducts quarterly Management Reviews, the review of CAPA status during a Management Review should focus on the most important issues rather than discuss every CAPA in detail. The impact on safety, the impact on product performance, and the economic impact of a specific CAPA are all criteria for deciding which CAPAs to discuss during a Management Review.

The CAPA Board needs a metric or metrics for monitoring the effectiveness of the CAPA process. The simplest metric is to monitor the average aging of CAPAs. If that average is steadily rising week after week, then new CAPAs are not being initiated, and existing CAPAs are not being closed. You can also measure the time to write a CAPA plan and the time to perform an investigation or monitor the on-time completion of tasks. The most important thing is for someone to take action when these metrics are not aligned with your quality objectives for the CAPA process. Taking action after 90 days of neglect is not good enough. You need to be monitoring the CAPA process weekly, and you need to take action proactively. Therefore, your CAPA Board needs to meet weekly and you need to show evidence in your CAPA records of what actions were taken by the CAPA Board.

Who should be assigned to the CAPA Board?

Top management does not need to be directly involved in the CAPA Board. Top management already reviews the status of CAPAs during Management Reviews. In a small company (i.e. < 20 people) you might have no choice but to have the same people that are assigned to your CAPA Board also be members of top management. As your company gets larger, you should assign middle-level managers and people that are new to management as members of the CAPA Board. Participating in the CAPA Board will teach those managers to work together as a team to achieve shared company goals and to persuade their peers to help them. The experience of working on a CAPA Board will also expose less experienced managers to other departments outside of their expertise. Ideally, participation in the CAPA Board will build friendships between peers that might not speak to one another. Each CAPA represents a team-building opportunity. The team needs to find a way to pool its resources to complete CAPAs on time and effectively. It is also important to rotate the assignment to the CAPA Board so that eventually all of your middle-level managers are trained in the CAPA process and each of them has been evaluated on their demonstration of team leadership and effectiveness in working with peers cooperatively. In large companies, it is common to assign one member of top management to the CAPA Board to show that top management is supportive of the CAPA process and to provide authorization for additional resources and funding for actions when needed. The top management representative should also be rotated to make sure that all of the top management remains competent in the CAPA process.

How does the CAPA Board manage the CAPA process?

The CAPA Board should never be blaming an individual or department for the lack of CAPA success. The CAPA Board should be anticipating when a CAPA is falling behind schedule or might not be as effective as it should be. Nobody on the team should be afraid to voice their opinion or to make a suggestion. Each member of the team has the responsibility of asking for help when they need it and asking for help as early as possible. The CAPA assignments should be shared between the team members, and one person should be responsible for chairing the meetings. If everyone is experienced in participating in CAPA Boards, then the role of the chairperson can be rotated each week. If one or more team members are inexperienced, the person on the CAPA Board assigned to training them should be teaching them how to participate in the meetings and prepare them for acting as chairperson.

Every CAPA Board meeting should have a planned agenda and meeting minutes. Every open CAPA should be discussed during the meeting, but the amount of time devoted to each CAPA should be adjusted for the risk of the CAPA failing to be completed on time or failing to be effective. If a CAPA is going smoothly, the discussion might only last seconds. Any discussion or actions planned that are specific to a CAPA should be documented in the individual CAPA record as well as the meeting minutes. This will ensure that the CAPA records are maintained as required by the ISO 13485 standard and the regulations.

What is a CAPA Board? and Do you need one? Read More »

Is monitoring every procedure required?

1 Comment / ISO 13485:2016, ISO Certification, Quality Management System / By

Process monitoring is required but do you know whether monitoring every procedure is required by the FDA QSR or ISO 13485?

One of the elements that Medical Device Academy has incorporated into each procedure we created in our turnkey quality system is a section titled, “monitoring and measurement.” The purpose of this section is to force each process owner to identify a process metric for monitoring every procedure. In some cases, we suggest a metric that would be appropriate for most companies establishing a new quality system. In other procedures, we use the following default text:

“Enter a quality metric that you want to track for this process in accordance with ISO 13485:2016, Clause 8.2.5 and the procedure for Monitoring, Measurement, and Analysis (SYS-017).”

Where are the requirements for process monitoring in 21 CFR 820?

Some of the companies that have purchased our turnkey quality system have asked, “Is it required to monitor and measure something in every procedure?” In general, it is not a specific requirement to have a metric specified in each procedure. In fact, if your quality system is not ISO 13485 certified, there are actually only a few places where the US FDA requires monitoring. The FDA does not have a section specific to monitoring and measurement of processes, but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used:

  • 21 CFR 820.100(a)(1) – “Analyzing processes, work operations, concessions, quality audit reports, quality records, service records, complaints, returned product, and other sources of quality data to identify existing and potential causes of nonconforming product, or other quality problems. Appropriate statistical methodology shall be employed where necessary to detect recurring quality problems;”
  • 21 CFR 820.200(b) – “Each manufacturer shall analyze service reports with appropriate statistical methodology in accordance with § 820.100.”
  • 21 CFR 820.250 – “(a) Where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics. (b) Sampling plans, when used, shall be written and based on a valid statistical rationale. Each manufacturer shall establish and maintain procedures to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed. These activities shall be documented.” Note: the other two instances are the title of 21 CFR 820.250.

The word “monitoring” is equally rare (i.e. 4x) in the QSR:

  • 21 CFR 820.70(a) – “Each manufacturer shall develop, conduct, control, and monitor production processes to ensure that a device conforms to its specifications…Where process controls are needed…(2) Monitoring and control of process parameters and component and device characteristics during production.”
  • 21 CFR 820.75(b) – “Each manufacturer shall establish and maintain procedures for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met…(2) For validated processes, the monitoring and control methods and data, the date performed, and, where appropriate, the individual(s) performing the process or the major equipment used shall be documented.”

Where are the requirements for process monitoring in ISO 13485:2016?

ISO 13485:2016 has a section specific to monitoring and measurement of processes (i.e. Clause 8.2.5). In addition, the word “monitoring” occurs 52 times in the standard and there are 60 incidents of some variant or the exact word. , but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used. There are four Clause headings that actually include the word monitoring:

  • Clause 7.6, Control of monitoring and measuring equipment
  • Clause 8.2, Monitoring and measurement
  • Clause 8.2.5, Monitoring and measurement of processes
  • Clause 8.2.6, Monitoring and measurement of product

In Clause 1, Scope, and Clause 4.1.5, the Standard states that any outsourced processes remain the responsibility of the company and must be accounted for in the quality system by monitoring, maintaining, and controlling the processes.

Monitoring of risk is included in the definition of “risk management” in the Standard (i.e. Clause 3.18).

Clause 4.1.3 states that the organization shall, “b) ensure the availability of resources and information necessary to support the operation and monitoring of these processes…d) monitor, measure as appropriate, and analyze these processes.”

Clause 4.2.3 states that the contents of the Medical Device File (i.e. MDR or TF), shall include, “d) procedures for measuring and monitoring.”

Monitoring and measurement of processes and product are required inputs to the Management Review in Clauses 5.6.2e) and f).

Clause 6.4.1 requires a procedure for monitoring the work environment if it can have an effect on product quality.

Clause 7.1 requires the company to consider including monitoring in product realization planning.

Clause 7.4.1 requires a plan for monitoring suppliers.

Clause 7.5.1 requires monitoring production and service, including the monitoring of process parameters and product characteristics.

Clause 7.5.6 requires monitoring of validated process parameters.

Clause 7.5.8 requires identification of status with regard to product monitoring and measurement (i.e. inspection status).

Clause 7.6 requires monitoring and measurement of calibrated devices and validation of any computer software used to monitor calibrated devices.

Clause 8.1 states that companies shall plan and implement monitoring and measurement of processes.

Clause 8.2 is titled, “Monitoring and measurement.”

Clause 8.2.1 requires monitoring of customer feedback.

Clause 8.2.5 requires monitoring of processes to ensure planned results are achieved.

Clause 8.2.6 requires monitoring of products to ensure product requirements have been met.

Clause 8.4 requires data analysis of monitoring data from at least six different processes:

  1. Feedback
  2. Conformity to product requirements
  3. Characteristics and trends of processes and products, including opportunities for improvement
  4. Suppliers
  5. Audits
  6. Service reports, as appropriate

In summary, while not every single clause that requires a procedure includes a requirement for monitoring, there are a number of processes where the requirement to monitor the process is explicitly stated.

Why do all of our procedures include the requirement for metrics?

Medical Device Academy expanded the requirement for monitoring to all procedures for five reasons:

  1. Quality objectives must be “established at relevant functions and levels within the organization.” Therefore, establishing monitoring requirements for each procedure ensures that top management has metrics for every process and a lack of data is never an excuse for not establishing a new quality objective when improvement is needed.
  2. If every procedure has a requirement for monitoring, then employees don’t have to remember which processes require monitoring and which processes do not explicitly require monitoring.
  3. The process approach to auditing includes metrics of the process as one of the seven items that are included in every process turtle diagram, and therefore, including metrics for each procedure facilitates the process approach to auditing.
  4. If a company does not have a process metric already established, it is often difficult to perform an investigation of the root cause of quality issues. If a metric is already being monitored for the process, this facilitates the investigation of the root cause and you can use the baseline monitoring data to help you establish effectiveness criteria for the corrective action.quantitative effectiveness check 300x209 Is monitoring every procedure required?
  5. Finally, most companies struggle to identify preventive actions as required by Clause 8.5.3, and we have found that data analysis of monitoring data is the best source of identifying new preventive actions.

What are the disadvantages when you monitor and measure something in every procedure?

The primary reason for resistance to identifying a metric for monitoring in every procedure is that it will increase the workload for the employees responsible for that process. However, monitoring of data does not always increase workload. In fact, when process data is recorded in real-time on a run chart it is often possible to identify a trend much earlier than when data is simply recorded and subjected to monitoring.

  • Example #1: The automatic tracking of toner in a printer tells HP when to ship you a new toner cartridge before you need it. This ensures that there is no loss in productivity because you never run out of ink or the ability to print documents.
  • Example #2: Companies will use project management software (e.g. Asana) to monitor labor utilization. This will help identify when a specific resource is nearing capacity. When this occurs, the project manager can add time buffers to prerequisite steps and adjust the starting date of the resource-limited tasks to an earlier starting date. This ensures that more time is available to finish the task or to take advantage of resource availability at an earlier date.
  • Example #3: Monitoring the revision date for procedures helps the document control process owner identify procedures that should be evaluated for the need to be revised and updated. Often this is articulated as a quality objective of reviewing and updating all procedures within 2 years. This also ensures that procedures remain current and compliant with regulatory requirements.

What are the advantages of monitoring every procedure?

The phrase “what gets measured gets managed” is a popular business philosophy that implies measuring employee activity increases the likelihood that employees will complete a task or perform it well. In contrast, if a process is not monitored, employees may assume that it is not important and the tasks may be skipped or completely forgotten. Setting quantitative goals is also sometimes integrated with economic incentives or bonuses that are granted to individuals and teams.

FDA transition from QSR to ISO 13485

The US FDA is planning its transition from 21 CFR 820 to ISO 13485 as the quality system criteria. This will force companies to make adjustments to their quality systems and increase the amount of process monitoring performed. My general advice is to work with employees that are performing tasks to identify streamlined methods for monitoring those tasks without being overly burdensome. Then you and the employees you manage can analyze the data together and identify opportunities for improvement. When you do this, experiment with manual methods using whiteboards and paper charts that are visible in public areas first. Only implement automated solutions after you have optimized the data being collected and the frequency of data collection, and remember that not every process will benefit from automated statistical process control. Sometimes the simple approach is best.

Is monitoring every procedure required? Read More »

What’s new in the 2022 draft cybersecurity guidance?

1 Comment / 510(k), Cybersecurity / By

On April 8, 2022, the FDA released a new draft cybersecurity guidance document to replace the 2018 draft that the industry does not support.

Why was the draft cybersecurity guidance created?

Due to the ubiquitous nature of software and networked devices in the medical industry, the impact of cybersecurity attacks is becoming more frequent and more severe. The WannaCry Ransomeware Attack is just one example of this global cybersecurity issue. The FDA is responding to the need for stronger cybersecurity controls by issuing a new draft cybersecurity guidance for 2022.

The first four paragraphs of the introduction explain why we need this, and WannaCry is mentioned in the second paragraph of the background section. This new guidance is only a draft, but this is the FDA’s third attempt at regulating the cybersecurity of medical devices. The first guidance was finalized in 2014. That’s the 9-page guidance we currently have in effect. The guidance mentions risk 11 times and there is no mention of testing requirements or a bill of materials (BOM). The 2018 draft guidance (24-pages) met with resistance from the industry for a lot of reasons. One of the reasons mentioned by Suzanne Schwartz in an interview is the inclusion of a cybersecurity bill of materials (CBOM). The industry felt it would be too burdensome to disclose all of the hardware elements that are related to cybersecurity. Therefore, the FDA rewrote the 2018 draft and released a new draft on April 8, 2022 (49-pages).

Untitled presentation e1650071404761 What’s new in the 2022 draft cybersecurity guidance?

You might have expected the FDA to soften its requirements in the face of resistance from industry, but the new draft does not appear to be less robust. It is true that the CBOM was replaced by a software bill of materials (SBOM). However, the SBOM must be electronically readable and it must include:

  • the asset(s) where the software resides;
  • the software component name;
  • the software component version;
  • the software component manufacturer;
  • the software level of support provided through monitoring and maintenance from the software component manufacturer;
  • the software component’s end-of-support date; and
  • any known vulnerabilities.

You can be sure that the medical device industry will view providing an SBOM as a hefty burden. After all, a machine-readable SBOM is more complex than UDI labeling requirements. An SBOM will not fit on the “Splash Screen” for anyone’s software application. Companies may provide documentation through the company website with a link in their software to that information. The format of the information could be in the “Manufacturer Disclosure Statement for Medical Device Security (MDS2).” However, MDS2 is a 349-line item Excel spreadsheet to be used as a checklist (i.e. quite a bit longer than the GUDID data elements spreadsheet), and it took the FDA eight years to complete the transition for the UDI Final Rule (i.e. 2013 – 2021).

The 2018 draft cybersecurity guidance document from the FDA required a cybersecurity bill of materials (CBOM). CBOM was defined as “a list that includes but is not limited to commercial, open source, and off-the-shelf software and hardware components that are or could become susceptible to vulnerabilities.” Therefore, the FDA’s change from a CBOM to an SBOM eliminated the requirement to disclose the hardware components. Despite the change in disclosure requirements, manufacturers will still be expected to monitor potential hardware vulnerabilities to cybersecurity attacks. It should also be noted that the language in the PATCH Act (a new bill submitted to the House of Representatives and to the Senate for ensuring the cybersecurity of medical devices) specifically requires manufacturers “to furnish a software bill of materials as required under section 524B (relating to ensuring the cybersecurity).”

 Structure of the draft cybersecurity guidance

The 2022 draft cybersecurity guidance organizes the requirements into four major principles:

  1. cybersecurity as part of device safety and the quality system regulations
  2. designing for security
  3. transparency
  4. submission documentation

The draft cybersecurity guidance recommends the implementation of a Secure Product Development Framework (SPDF). However, there is not much detail provided in the guidance for a SPDF. In the past, the term for this type of process was referred to as a Secure Software Development Lifecycle (i.e. Secure SDLC). However, in February 2022, the NIST Computer Security Resource Center (CSRC) released version 1.1 of the Secure SDLC guidance which is now titled “Secure Software Development Framework (SSDF) Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities.” This guidance provides guidance on the implementation of best practices for reducing the risk of software vulnerabilities because existing standards for managing the software development lifecycle do not explicitly address software security (e.g. IEC 62304-1:2015). The SSDF recommends implementing a core set of high-level secure software development practices that can be integrated into your SDLC process. Your software development team will also require cybersecurity training.

Design for security is the second principle of the draft cybersecurity guidance

Under this new draft cybersecurity guidance, the FDA will be evaluating the cybersecurity of devices based on the ability of the device to provide and implement the following security objectives:

  • Authenticity, which includes integrity;
  • Authorization;
  • Availability;
  • Confidentiality; and
  • Secure and timely updatability and patchability.

Transparency of cybersecurity information is for users

The draft cybersecurity guidance seeks to give device users more information pertaining to the device’s cybersecurity controls, potential risks, and other relevant information. This information will be in the form of an SBOM that is electronically readable. This information shall include disclosure of 1) known vulnerabilities or risks, 2) information to securely configure and update devices, and 3) communication interfaces and third-party software.

In addition to providing an SBOM, the FDA draft cybersecurity guidance includes requirements for cybersecurity labeling in section VI(A). There are 15 specific labeling requirements identified by the FDA for sharing with device users to improve the transparency of cybersecurity information. The first of these requirements is recommendations from the manufacturer for cybersecurity controls appropriate for the intended use environment (e.g., antimalware software, use of a firewall, password requirements). This first labeling requirement is identical to the 2018 draft guidance. Several of the other requirements are copied from the 2018 draft guidance, but others are new and/or reworded cybersecurity labeling requirements.

FDA Submission Documentation Requirements

The 2022 FDA draft cybersecurity guidance includes requirements for FDA submission documentation. Submission documentation must include a security risk management plan and report. The draft cybersecurity guidance explains on page 13 (numbered 9) that “performing security risk management is a distinct process from performing safety risk management as described in ISO 14971:2019.” Therefore, instead of using your safety risk management process, your software development team will need to have a different risk management process for software security. Details on the content for security risk management plans and reports can be found in AAMI TIR57:2016 – Principles for medical device security—Risk management. Appendix 2 also provides guidance for the inclusion of a) call flow diagrams, and b) information details for an architecture view.

Cybersecurity testing requirements for your FDA submission

The biggest impact of this new draft guidance may be the requirement for testing. The 2014 guidance has no testing requirement, the 2018 draft guidance mentioned testing 5 times in a few bullet points, but this new draft guidance mentions testing 43 times. The testing requirements for cybersecurity risk management verification include:

  1. Security requirements
  2. Threat mitigation
  3. Vulnerability testing
  4. Penetration testing

This guidance also includes a paragraph with multiple bullets of requirements for each of the four types of testing. This would essentially double the size and scope of the current software section for a 510k submission, and manufacturers will need to create new procedures and templates for their cybersecurity risk management process. For example, penetration testing requirements include the following elements:

  • Independence and technical expertise of testers,
  • Scope of testing,
  • Duration of testing,
  • Testing methods employed, and
  • Test results, findings, and observations.

Differences between the cybersecurity guidance documents

The following table provides a high-level overview comparing the four cybersecurity guidance documents released by the FDA, including the 2016 guidance on post-market management of cybersecurity:

Screenshot 2022 04 16 12.48.51 AM 1024x291 What’s new in the 2022 draft cybersecurity guidance?

Vulnerability management plans

The FDA draft cybersecurity guidance document also has a requirement for manufacturers to develop a plan for identifying and communicating vulnerabilities to device users after the release of the device. The FDA requires this plan to be included in your device submission. The vulnerability management plan should include the following information (in addition to the requirements of the 2016 guidance for postmarket cybersecurity management):

  • Personnel responsible;
  • Sources, methods, and frequency for monitoring for and identifying vulnerabilities (e.g. researchers, NIST NVD, third-party software manufacturers, etc.);
  • Periodic security testing to test identified vulnerability impact;
  • Timeline to develop and release patches;
  • Update processes;
  • Patching capability (i.e. rate at which update can be delivered to devices);
  • Description of their coordinated vulnerability disclosure process; and
  • Description of how manufacturer intends to communicate forthcoming remediations, patches, and updates to customers.

What’s the next step for the draft cybersecurity guidance?

In March the “Protecting and Transforming Cyber Health Care Act of 2022 (PATCH Act)” was introduced to the House of Representatives and the Senate. The goal of the PATCH Act is to enhance medical device security by requiring manufacturers to create a cybersecurity risk management plan for monitoring and addressing potential postmarket cybersecurity vulnerabilities. The FDA seeks comments on the draft cybersecurity guidance through July 7, 2022. Given the support of the new bill in the House of Representatives and Congress, it is likely that the FDA will get the support it needs for this new guidance. 

What’s new in the 2022 draft cybersecurity guidance? Read More »

What are the IVDR risk management requirements?

3 Comments / ISO 14971:2019 (Risk Management), IVDR - Regulation (EU) 2017/746 / By

This article reviews unique IVDR risk management requirements for CE Marking of in vitro diagnostic (IVD) devices in Europe.

Last week I posted a blog about “How to create an IVDR checklist.” The article was very popular because we included a form for downloading a free IVDR checklist. That form included the opportunity for people to ask a question about the IVDR. One of the subscribers, a gentleman from New Zealand, entered a very simple comment: “risk management requirements.” My first thought was that the risk management file is the required technical documentation for the IVDR. Then I quickly remembered that in 2012, EN ISO 14971:2012 was released with three new annexes for the three directives: ZA (for the MDD), ZB (for the AIMD), and ZC (for the IVDD). In Annex ZC there were seven deviations, and even though ISO 14971 was updated in 2019, the international standard continues to deviate from the European regulations in significant ways. Therefore, this blog provides an overview of the IVDR risk management requirements.

If you are already compliant with ISO 14971:2019, do you meet the IVDR risk management requirements?

The biggest difference between the ISO 14971:2019 standard and the IVDR risk management requirements is that the standard only requires a benefit-risk analysis to be performed if risks are unacceptable. In contrast, the IVDR requires that a benefit/risk analysis be performed for all risks and the overall residual risk. Therefore, you must include a benefit/risk analysis in your technical file submission regardless of risk acceptability. The harmonized version of the standard (i.e. EN ISO 14971:2019/A:11:2021) was released in December of 2021. If you already purchased ISO 14971:2019, you only need to purchase the amendment which consists of Annex ZA (comparison between the standard and Annex I of MDR) and ZB (comparison between the standard and Annex I of the IVDR).

In the amendment, it states that manufacturers must have a risk management policy that is compliant with Annex I of the EU regulation. There are notes at the beginning of each harmonization annex that indicates that the risk management process needs to be compliant with the IVDR, which means risks have to be ‘reduced as far as possible’, ‘reduced to a level as low as reasonably practicable’, ‘reduced to the lowest possible level’, ‘reduced as far as possible and appropriate’, ‘removed or reduced as far as possible’, ‘eliminated or reduced as far as possible’, ‘prevented’ or ‘minimized’, according to the wording of the corresponding section in Annex I of the IVDR. The comparison table has a column with remarks/notes. In most cases, the deficiency identified states, “Device-specific execution of the process is not covered.” There are also two remarks/notes that state “Device-specific and usability-specific execution of the process is not covered.” 

Where are the IVDR risk management requirements?

Blog tip 1024x183 What are the IVDR risk management requirements?

IVDR risk management requirements are found in Annex II, Section 5 of the IVDR. However, there are 228 references to the word risk throughout the IVDR. The following risk-related requirements in IVDR are particularly important:

  • Article 10, Sections 2 & 8(e) – risk management procedure requirement
  • Annex I, Section 3 – reiteration of risk management procedure requirement, but specific steps in the risk management process are identified (e.g. a risk management plan, hazard identification, estimation of risks, evaluation of risks, etc.)
  • Annex I, Section 4 – Priority of risk control measures
  • Annex I, Section 5 – Elimination or reduction of use-related risks
  • Annex III, Section 1(b) – Reassessment of the benefit-risk analysis and risk management using post-market surveillance data

How should you document your risk management file for the IVDR?

In your risk management file of course. There is no format requirement for risk management files, but there are requirements for the content and there is a GHTF guidance document for risk management, and ISO/TR 24971:2020 is a new guidance document on the application of ISO 14971 to medical devices. Neither of these guidance documents is specific to IVDR risk management requirements. Annex H of ISO/TR 24971:2020, however, provides guidance specific to IVD devices.

What do the IVDR risk management requirements include for risk analysis?

In our March 23, 2022 blog posting, I described four types of risk analysis:

  1. Design risk analysis
  2. Process risk analysis
  3. Software hazard analysis
  4. User-related risk analysis (URRA)

Of these four types of risk analysis, only the software hazard analysis is sometimes not applicable. For an FDA 510(k) submission, you would need to provide software hazard analysis and URRA in the actual submission. The other two types of risk analysis would only be included in your design history file (DHF), and the FDA would review the design and process risk analysis during a routine inspection when the DHF is sampled as part of the design control process.

In contrast, the IVDR requires that a complete risk management file be submitted as part of the technical file (see Annex II, Section 5):

“Benefit-risk analysis and risk management

The documentation shall contain information on:

    • the benefit-risk analysis referred to in Sections 1 and 8 of Annex I, and
    • the solution adopted and the results of the risk management referred to in Section 3 of Annex I.”

The above documentation typically consists of design risk analysis and does not typically include process risk analysis, software hazard analysis, or use-related risk analysis. These other three risk analysis documents are IVDR risk management requirements, but they are referenced by the technical file in other sections. The most obvious IVDR risk management requirements are referenced in Annex I, Sections 1-9. These are referred to as the General Safety and Performance Requirements (GSPRs), and this requirement is typically met by including a GSPR checklist in the technical file to meet the requirement of Annex II, Section 4.

The process risk analysis is typically included with manufacturing information to meet the requirement of Annex II, Section 3.2. This documentation may include, any and all of the following elements:

  1. a process failure mode and effects analysis (pFMEA)
  2. a risk control plan including all processes from receiving inspection to final inspection and product release
  3. a process validation plan that is risk-based and linked to the risk control plan

The best practice for estimation of process risks is to link the probability of occurrence and probability of detection to the quantitative data gathered during process validation. In addition, you may establish a risk management policy that prescribes specific types of process risk controls (e.g. automated inspection) for the highest risk processes where manufacturing process errors are not acceptable residual risks. For example, an inspection of printed circuit board assemblies (PCBAs) typically requires automated optical inspection (AOI) methods, because visual inspection is not sufficient by itself and not all PCBAs allow sufficient ICT coverage, and functional testing is limited.

The software hazard analysis, if applicable, is typically performed in accordance with IEC/TR 80002-1:2009, Guidance on the application of ISO 14971 to medical device software. In the software hazard analysis, it is unnecessary to estimate the probability of occurrence of harm. Instead, it is only necessary to identify hazards and estimate harm. Examples of these hazards include loss of communication, mix-up of data, loss of data, etc. Software failures are systemic in nature and the probability of occurrence cannot be determined using traditional statistical methods. Therefore, we recommend that you assume that the failure will occur and estimate software risks based on the severity of the hazard resulting from the failure. For these reasons, it is recommended that software hazard analysis documentation is maintained as a separate document from your design risk analysis. The software hazard analysis documentation should be referenced in your risk management report, but the software hazard analysis should be included as part of your software verification and validation. The IVDR requires that you include a summary of software verification and validation in Annex II, Section 6.4 rather than the complete hazard analysis document.

A use-related risk analysis should be part of your useability engineering file for IVD devices as required by EN 62366-1:2015. Use-related risks are mentioned in Annex I, Section 5:

“In eliminating or reducing risks related to use error, the manufacturer shall:

    • reduce as far as possible the risks related to ergonomic features of the device and the environment in which the device is intended to be used (design for patient safety), and
    • Give consideration to the technical knowledge, experience, education, training and use environment, where applicable, and the medical and physical conditions of intended users (design for lay, and professional, disabled or other users).”

The above requirement includes not only the ability to read and interpret test results of IVD devices but also the ability of laypersons to properly self-select if an IVD is intended to be sold as an over-the-counter product. Usability also is mentioned in Article 78, Section 3(f):

“for the identification of options to improve the usability, performance and safety of the device;”

Therefore, there should be specific elements of your post-market surveillance plan that are designed to gather feedback on the usability of your IVD device. 

When should risk management activities be performed for IVD devices?

The IVDR does not specifically define when in the design and development process the various risk management activities shall be performed. However, the required risk management activities are specified in the IVDR within Annex I. ISO 14971:2019, however, is more descriptive of the risk management activities and the risk management process. Therefore, your risk management plan should align with the process defined in ISO 14971:2019, Clause 4.1.

Unfortunately, most companies do not include risk management as an integral part of the design and development process. Instead, risk management documentation is created retroactively as part of the documentation preparation for technical file submission. For this reason, most medical device executives fail to see the benefit associated with the risk management process. Even biomedical engineers struggle to appreciate the necessity of following the process outlined in the risk management standard in order to prevent device malfunctions and use errors.

The following is a list of the required risk management activities in the order that they should be occurring. Each activity also references the applicable clause of ISO 14971:2019. We have also grouped the activities into the five phases of design and development:

Design Controls with risk 1024x542 What are the IVDR risk management requirements?

Design Planning

    • Risk management planning (Clause 4.4)

Design Inputs

    • Identification of hazards and hazardous situations (Clause 5.4)

Design & Development

    • Risk estimation (Clause 5.5)
    • Risk evaluation (Clause 6)
    • Risk control option analysis (Clause 7.1)
    • Implementation of risk control measures (Clause 7.2)
    • Residual risk evaluation (Clause 7.3)

Design Verification and Validation

    • Benefit/risk analysis (Clause 7.4)
    • Risk control effectiveness verification (Clause 7.6)

Design Release

    • Evaluation of overall residual risk (Clause 8)
    • Risk management review (Clause 9)

If your company is preparing a 510(k), the company may be able to submit the 510(k) immediately after completion of risk control effectiveness verification. You may also be able to postpone the benefit/risk analysis until you submit your IVD technical file for CE Marking approval. The benefit/risk analysis is not required by ISO 14971 unless the risks are unacceptable, and the FDA does not require a benefit/risk analysis except for novel devices seeking market authorization through a De Novo Classification Request or a Pre-Market Approval (PMA). The FDA also does not require the submission of the complete risk management file.

IVDR risk management requirements are quite different than the US FDA requirements for risk management. An IVD technical file must include a risk management summary report that summarizes all activities that were performed according to the risk management plan. A benefit/risk analysis is required for each risk and the overall risk. The Notified Body auditor is also expected to sample the complete risk management file during quality system audits. Finally, the IVDR includes a requirement for a post-market surveillance plan that includes the collection of production and post-production data as feedback on the risk management process and a post-market clinical performance follow-up (PMPF) plan. 

What production and post-production information should you be collecting for IVD devices?

Medical device manufacturers struggle to see the benefits of requiring a post-market surveillance system, and smaller companies, in particular, complain that the cost of the new European post-market surveillance requirements is excessive and prohibits innovation. However, the primary role of post-market surveillance is to ensure rapid initiation of containment and corrective actions for devices that malfunction and/or present unacceptable risks to the intended users and intended patient population. The purpose of generating the post-market surveillance data is defined in the IVDR within Article 78, Section 3.

The minimum requirements for post-market surveillance are defined in Annex III, Section 1(a):

  • Information concerning serious incidents, including information from PSURs, and field safety corrective actions;
  • records referring to non-serious incidents and data on any undesirable side-effects;
  • information from trend reporting;
  • relevant specialist or technical literature, databases and/or registers;
  • information, including feedback and complaints, provided by users, distributors, and importers; and
  • publicly-available information about similar medical devices.

The IVDR is not prescriptive regarding what production data shall be collected for post-market surveillance, but the reason for this is that there are many different types of manufacturing processes with different process risks. In addition, the IVDR includes software as a medical device where there is no manufacturing process at all. Therefore, the best approach for determining what production data to collect is the review your process risk analysis (e.g. pFMEA). The process risk analysis for each manufacturing process should allow you to identify the manufacturing process steps that have the greatest residual risks (e.g. risk priority number or RPN) and potentially the highest severity of the effect. The risks should be identified as a priority for post-market surveillance. You should also include process parameter monitoring data for any validated processes (e.g. sterilization time, temperature, and pressure). Finally, you should also monitor rejects at incoming inspection, in-process inspection, and final inspection operations.  

Other IVD Risk Management Resources

The following resources may be helpful for creating and maintaining your IVD risk management file:

  1. EN ISO 14971:2019 + A11:2021
  2. ISO/TR 24971:2020
  3. GHTF/SG3/N15R8
  4. Regulation (EU) 2017/746 (i.e. IVDR)
  5. IEC/TR 80002-1:2009
  6. EN 62366-1:2015 + A1:2020

Note: Whenever possible, hyperlinks to the Estonian Centre for Standardization and Accreditation (EVS) are provided for procedures, because we find that this source is frequently the least expensive, and digital versions are available on-demand as a multi-user license.

What are the IVDR risk management requirements? Read More »

How to create an IVDR checklist

4 Comments / CE Marking, In Vitro Diagnostic (IVD) Devices, ISO 13485:2016, ISO Auditing, ISO Certification, IVDR - Regulation (EU) 2017/746 / By

This article provides an IVDR checklist for updating your ISO 13485 quality system to comply with EU Regulation 2017/746.

IVD Checklist 1024x474 How to create an IVDR checklist

Why I created an IVDR checklist?

Hundreds (if not thousands) of IVD manufacturers are currently updating their ISO 13485:2016 certified quality system from compliance with the In Vitro Diagnostic Directive (i.e. Directive 98/79/EC) or IVDD to the new EU In Vitro Diagnostic Regulation (i.e. Regulation 2017/746). Revision of technical files and the associated procedures for creating your technical files is a big part of these updates. However, there is much more that needs to be updated than just the technical documentation. Therefore, IVD manufacturers are asking Medical Device Academy to conduct remote internal audits of their quality system to identify any gaps. Usually, we conduct internal audits using the process approach to auditing, but this is one of the scenarios where the element approach and an audit checklist are invaluable.

If you would like to download our IVDR checklist for FREE, please fill in the form below.

How do you use an audit checklist?

An audit checklist is used by quality system auditors to collect objective evidence during an audit. This objective evidence verifies compliance with regulatory requirements or internal procedural requirements. If the auditor is unable to find supporting evidence of compliance, the auditor may continue to search for data or identify the requirement as a nonconformity. Typically the checklist is in four columns using a tabular form. The left-hand column lists each requirement. The next column is where the auditor documents records sampled, procedures reviewed, and personnel interviewed. In the third column, the auditor indicates what they were looking for in the records, procedures, or during the interview. Some of the information in the second and third columns can often be entered prior to starting the audit by reviewing audit preparation documents (e.g. procedures and previous audit reports). In the fourth column the auditor will enter the objective evidence for conformity collected during the audit.

How to create an IVDR quality plan

Most of the companies that are preparing for an IVDR audit by their notified body already have ISO 13485:2016 certification and they are using the self-declaration pathway for CE Marking under the IVDD. Under the IVDR, a notified body must now review and approve the technical file. The notified body must also confirm that their quality system has been updated to include the IVDR requirements. The Technical File requirements are found in Annex II and III; while most of the quality system requirements are found in the Articles.  The quality system requirements include:

  1. a risk management process in accordance with Annex I – deviations from ISO 14971:2019 will be necessary)
  2. conduct a performance evaluation–including a post-market performance follow-up (PMPF). This requirement is defined in Article 52 and Annex XIII
  3. create and maintain a technical file in accordance with Annex II & III
  4. create and maintain a Declaration of Conformity in accordance with Article 17
  5. CE Mark the product in accordance with Article 18
  6. implement a UDI system in accordance with Article 24, 26, and 28
  7. record retention requirements for the technical file, Declaration of Conformity, and certificates shall be increased from 5 years to 10 years
  8. set-up, implement, and maintain a post-market surveillance system in accordance with Article 78
  9. document a procedure for communication with Competent Authorities, Notified Bodies, Economic Operators, Customers, and/or other Stakeholders
  10. update procedures for reporting of serious incidents and field safety corrective actions in the context of vigilance to require reporting within 15 calendar days
  11. update the product labeling to comply with Annex I, section 20
  12. revise the translation procedure to ensure translations of the instructions for use are available in all required languages of the member states, and make sure these translations are available on the company website
  13. create a procedure for utilization of the Eudamed database for registration, CE Marking applications, UDI data entry, and vigilance reporting

Which IVDR requirements are already met by your quality system?

Some companies also manufacture medical devices that must comply with Regulation (EU) 2017/745. For those companies, many of the above requirements are already incorporated into their quality system. In this case, you should still include all of the IVDR checklist requirements in your plan, but you should indicate that the requirement has already been met and audited previously.

Content related to our IVDR checklist

On Friday, April 1, 2022 @ 11 am EDT (8 am Pacific), Rob Packard will be Joe Hage’s guest speaker on the weekly MDG Premium Live video (please click on the link to register). The topic of the live presentation will be “How to create an IVDR quality plan.” #MedicalDevices #MDGpremium

How to create an IVDR checklist Read More »

What are the four types of risk analysis?

1 Comment / ISO 14971:2019 (Risk Management) / By / , , , , , , , , , , , ,

You are familiar with design and process risk analysis, but do you know all four types of risk analysis?

Last week’s YouTube live streaming video answered the question, “What are the four different types of risk analysis?” Everyone in the medical device industry is familiar with ISO 14971:2019 as the standard for medical device risk management, but most of us are only familiar with two or three ways to analyze risks. Most people immediately think that this is going to be a tutorial about four different tools for risk management (e.g. FMEA, Fault Tree Analysis, HAZOP, HACCP, etc.). Instead, this article is describing the four different quality system processes that need risk analysis.

What are the four different types?

The one most people are familiar with is risk analysis associated with the design of a medical device. Do you know what the other three are? The second type is process risk management where you document your risk estimation in a process risk analysis. The third type is part of the medical device software development process, specifically a software hazard analysis. Finally, the fourth type is a Use-Related Risk Analysis (URRA) which is part of your usability engineering and human factors testing. Each type of risk analysis requires different information and there are reasons why you should not combine these into one risk management document or template.

Design Risk Analysis

Design risk analysis is the first type of risk analysis we are reviewing in this article. The most common types of design risk analysis are the design failure modes and effects analysis (dFMEA) and the fault-tree analysis (FTA). The dFMEA is referred to as a bottom-up method because you being by identifying all of the possible failure modes for each component of the medical device and you work your way backward to the resulting effects of each failure mode. In contrast, the FTA is a top-down approach, because you begin with the resulting failure and work your way down to each of the potential causes of the failure. The dFMEA is typically preferred by engineers on a development team because they designed each of the components. However, during a complaint investigation, the FTA is preferred, because you will be informed of the alleged failure of the device by the complainant, but you need to investigate the complaint to determine the cause of the failure. Regardless of which risk analysis tool is used for estimating design risks, the risk management process requires that production and post-production risks be monitored. Therefore, the dFMEA or the FTA will need to be reviewed and updated as post-market data is gathered. If a change to the risk analysis is required, it may also be necessary to update the instructions for use to include new warnings or precautions to prevent use errors.

Process Risk Analysis

Process risk analysis is the second type of risk analysis. The purpose of process risk analysis is to minimize the risk of devices being manufactured incorrectly. The most common method of analyzing risks is to use a process failure modes and effects analysis (i.e. pFMEA). This method is referred to as a bottom-up method because you begin by identifying all of the possible failure modes for each manufacturing process step. Next, the effects of the process failure are identified. After you identify the effects of failure for each process step, the severity of harm is estimated. Then the probability of occurrence of harm is estimated, and the ability to detect the failure is estimated. Each of the three estimates (i.e. Severity, Occurrence, and Detectability) are multiplied to calculate a risk priority number (RPN). The resulting RPN is used to prioritize the development of risk controls for each process step.

As risk controls are implemented, the occurrence and detectability scores estimated again. This is usually where people end the pFMEA process, but to complete one cycle of the pFMEA the risk management team should document the verification of the effectiveness of the risk controls implemented. For example, if the step of the process is sterilization then documentation of effectiveness consists of a sterilization validation report. This is the last step of one cycle in the pFMEA, but the risk management process includes monitoring production and post-production risks. Therefore, as new process failures occur the pFMEA is reviewed to determine if any adjustments are needed in the estimates for severity, occurrence, or detectability. If any of the risks increase, then additional risk controls may be necessary. This process is continuously updated with production and post-production information to ensure that process risks remain acceptable.

Software Hazard Analysis

Sofware hazard analysis is becoming more important to medical devices as physical devices are integrated with hospital information systems and with the development of software as a medical device (SaMD). Software risk analysis is typically referred to as hazard analysis because it is unnecessary to estimate the probability of occurrence of harm. Instead, it is only necessary to identify hazards and estimate harm. Examples of these hazards include loss of communication, mix-up of data, loss of data, etc. For guidance on software hazard identification, IEC/TR 80002-1:2009 is a resource. FDA software validation guidance indicates that software failures are systemic in nature and the probability of occurrence cannot be determined using traditional statistical methods. Therefore, the FDA recommends that you assume that the failure will occur and estimate software risks based on the severity of the hazard resulting from the failure. 

Use-Related Risk Analysis

The fourth and final type of risk analysis is use-related risk analysis (URRA). Most development teams assume that they are able to use traditional hazard identification techniques to identify the potential use-related risks. However, use-related risks are inextricably linked to the experiences of the user. The development team has unique knowledge of the device they are developing, and therefore it is likely that use-related risks associated with a lack of knowledge about the device will result in use errors that the development team would not realize. For this reason, formative testing is necessary to identify unforeseen use-related risks. Once formative testing identifies these risks, additional formative usability testing can be used to create and refine the instructions for the use of a medical device. Finally, formative testing can be used to develop user training programs that prevent potential use errors. Once the development team has completed the necessary formative testing, then summative usability testing is used to validate the effectiveness of the risk controls that were implemented.
 
In the past, I believed that the FDA’s focus on usability was the review of summative usability testing. However, I have learned that the FDA feels it is equally important to begin the human factors testing process by first performing a use-related risk analysis and then identifying the critical tasks. Without identifying these critical tasks, it is not possible for the FDA to determine if the moderator of the summative testing has observed all of the critical tasks being performed correctly. An example of a Use-Related Risk Analysis (URRA) was provided by the FDA in a 510(k) AI deficiency letter that we received. The example is provided below.
URRA table example from the FDA 300x117 What are the four types of risk analysis? Example of a URRA Table provided by the FDA
 

Can you use only the IFU to prevent use-related risks?

Instructions for use (IFU) are required to include warnings and precautions. This information provided by the manufacturer explains how to use a medical device correctly and identifies the residual risks. This is a form of risk control, but it is the least effective form of risk control and should be the risk control of last resort. Not everyone reads the IFU, and you cannot guarantee that everyone will understand the instructions. You certainly can’t be sure that users will remember all your warnings or precautions when they are tired, stressed, or acting in an emergency situation. Design controls and protective measures should be implemented as the first and second priority for risk controls, and the IFU should be your lowest priority.
 
This is the reason why we have color-coding, design features that eliminate the possibility of a use error, we provide training to users, and we are required to monitor use-related risks for medical devices. Formative usability testing is intended to identify use errors we did not anticipate, to help us develop instructions for use (IFU), and help us develop training for users. Summative testing is intended to validate that the design, training, and IFU are effective at preventing use errors. All three of these aspects work together–not the IFU alone. In fact, there is an entire alarms standard that identifies protective measures that shall be used for electromedical devices to prevent use errors (i.e. – IEC 60601-1-8).

 

Facilitating Risk Management Activities – An Interview with Rick Stockton

I listened to our YouTube video about the four different types of risk analysis, you may have heard my reference to Rick Stockton’s interview that we posted on our YouTube channel and embedded above. In our interview with Rick Stockton, we discussed how to facilitate risk management activities during the design and development of medical devices. If you are interested in learning more about Rick and facilitating risk management activities, please watch the video of our interview with Rick.
 

What are the four types of risk analysis? Read More »

Scroll to Top