Blog

Archive for Uncategorized

CAPA procedure, How do you improve quality and prevent nonconformity?

Your CAPA procedure is the most important SOP. It forces you to investigate quality problems and take actions to prevent nonconformity.

Your cart is empty

CAUTION: Read the story in the next few paragraphs before you implement any purchased procedure

During a recent internal audit, I noticed that the client was not meeting one of the requirements of their CAPA procedure. Specifically, the procedure indicated that all CAPA plans must be written within seven calendar days of initiating the CAPA. Despite this requirement in their procedure, the client was indicating that CAPA plans were due within 30 calendar days on their CAPA form.

This example is a minor nonconformity, but the reason why this client was not following their procedure is more interesting. The procedure was 100% compliant with FDA regulations, but the procedure did not match how the company performed the process. The procedure and the process MUST match.

This client purchased their CAPA procedure from another consultant, changed the title, and had everyone in the company “read and understand” the procedure for training.

Make sure your CAPA procedure is clear and concise

Procedures are often unclear because the author is more familiar with the process than the intended audience for the procedure. An author may abbreviate a step or skip it altogether. As an author, you should use an outline format and match your CAPA form exactly. There should be nothing extra in the procedure, and nothing left out. Medical Device Academy’s updated CAPA procedure is only six pages and the CAPA form is four pages.

Corrective and Preventive Action CAPA Procedure CAPA procedure, How do you improve quality and prevent nonconformity?
SYS-024 Corrective and Preventive Action (CAPA) Procedure, Form, and Log

SYS-024 – Medical Device Academy’s newly updated CAPA procedure is a 6-page procedure. Your purchase will also include our CAPA form (FRM-009), and our CAPA log (LST-005). The procedure is compliant with ISO 13485:2016, 21 CFR 820.100, SOR 98/282, and the EU MDR. You will also receive free updates in the future. We are currently distributing our 16th version of the procedure.

Price: $299.00

CAPA procedure writing recommendations

Procedures are often unclear because the author is an expert with more experience than the intended audience. An author may abbreviate a step or skip it altogether. As an author, you should use an outline format and match your CAPA form exactly. There should be nothing extra in the procedure, and nothing left out. Before writing your own CAPA procedure, consider following these 7 steps:

  1. Design your CAPA form first
  2. Identify which steps in your process are most important and specify how these steps will be monitored (i.e., risk-based approach)
  3. Write a procedure that follows your CAPA form and includes instructions for monitoring and measuring your CAPA process
  4. Conduct group CAPA training using the draft version of your form and procedure
  5. Make revisions to the form and procedure to clarify steps the trainees had difficulty with
  6. Ask the trainees to review the revised form and procedure
  7. Make final revisions and route the procedure for approval

The specific order of steps is essential to creating a CAPA procedure—or any procedure. Writing a procedure that matches the form used with that procedure helps people understand the tasks within a process. Throughout the rest of this article, we describe each of the nine steps of Medical Device Academy’s CAPA procedure (SYS-024). The actual CAPA form (FRM-009) sold with SYS-024 is more complex than 9 steps, but a more complex form is needed to make sure every sub-task is documented in your CAPA records.

Review nonconformities, including complaints, to determine if a CAPA is needed (Step 1)

If I am auditing a CAPA process, and almost all the CAPAs are resulting from auditor findings, then I know the client is not adequately reviewing other sources of potential quality issues. When I took my first CAPA training course, the image below was drawn on a flip chart by Kim Trautman. I have used this image in all of my CAPA training for other people since. I think this provides a good visual representation of the most common sources of new CAPAs. Although the number of CAPAs from each source will never be equal, you should review all of these data sources for quality issues periodically. 

Risk based CAPA Process Diagram 1024x465 CAPA procedure, How do you improve quality and prevent nonconformity?

CAPA procedure step 2 – Describe and reference the quality issue

The next step is to copy and paste your quality issue directly into the CAPA record and add a reference to the source. This step of the CAPA procedure is not a specific requirement of the ISO 13485 standard or the FDA regulations. However, describing and referencing quality issues in your CAPA record is a practical requirement. The person assigned to investigate the root cause of the quality issue needs to know what the source of the quality issue is, and when you are trying to close a complaint or audit report you will find it helpful to cross-reference the two records. For example, the CAPA might be related to the fifth nonconformity in your second internal audit report for 2022 (e.g. IA 220205).

Copy and Paste 300x146 CAPA procedure, How do you improve quality and prevent nonconformity?

Copy & Paste into your CAPA Record

Attribution: Icons were copied and pasted from Flaticon.com.

Step 3 – Perform a root cause analysis

Why can’t we fix our mistakes the first time? We are doomed to repeat mistakes when we fail to identify the root cause or causes. The person you assign to investigate a quality issue must be trained to perform a thorough root cause analysis. Successful root cause analysis depends upon four things:

  1. Courage to admit that your process is broken
  2. Learning more than one tool for analyzing problems
  3. Practicing the use of root cause analysis tools
  4. Sampling enough records (or testing enough product)

The common belief is that people fail to identify the root cause because they need root cause training (#2) or more practice (#3). However, most people fail because they stop sampling or testing too soon. I typically recommend that companies sample at least twice as many records as the suspected problem frequency. For example, if a complaint occurs 1% of the time, you should review 200 records before you can be sure you identified that root cause. If you are correct, you will only find the quality issue twice in the 200 records. However, a review of 200 records often reveals that the quality problem is more common than you originally estimated and there is more than one cause of device malfunction.

CAPA procedure step 4 – Do you need a new CAPA?

After you have successfully completed a root cause analysis, you now need to determine if a new CAPA is needed. If there is already a CAPA that is open for the same quality issue, you can use the existing CAPA as justification for not conducting corrective actions. In this case, you should include a cross-reference in the new CAPA record to the existing CAPA record. You should also document containment measures and corrections.

In both the existing CAPA record and your new CAPA record you should also be documenting your risk evaluation of your CAPA. In the latest update to our CAPA procedure, we changed the method of risk evaluation to match the MDSAP grading process for nonconformities. A copy of this section is provided in the image below.

CAPA Procedure Risk Evaluation Section 1024x231 CAPA procedure, How do you improve quality and prevent nonconformity?

In the CAPA procedure, we state that any risk score of 4 or 5 requires the implementation of a CAPA. If any of the escalation rules apply to the risk score, you should implement a CAPA regardless of the total risk score. This is our recommendation for a method of risk evaluation, but there is no standard telling you that you must do it this way. However, we believe this method of calculation is more likely to be consistent because it is based on the MDSAP grading guidelines.

If no escalation rules apply to your risk score, it may be possible to implement containment and corrections only. If your action plan includes only containment and corrections, we recommend that you monitor the quality issue as a process metric or quality objective to identify future occurrences. If you are evaluating a new CAPA that appears to have the same root cause as an existing CAPA, you may need to update the risk score of the existing CAPA to a higher number based on the escalation rules. Escalation may impact your corrective action plan, and it should certainly affect the prioritization of your existing CAPA.

Plan and document your corrective actions, including updating documentation (Step 5)

The biggest mistake you can make in this stage of the CAPA process is to spend too much time planning your corrective actions. “Take action and document it” is the essence of this step in the CAPA procedure. If you spend all of your time planning, then you will never take action. The CAPA plan can and should be edited. Therefore, if you know a procedure needs revision, start revising the procedure immediately. You can always add more corrective actions to your corrective action plan after you write the procedure, but you need to start writing. The second biggest mistake you can make during this stage is failing to document the actions you take. If you don’t document your actions, it’s a rumor, not a record.

Take Action and Document It 1024x121 CAPA procedure, How do you improve quality and prevent nonconformity?

Do your planned actions adversely affect regulatory compliance or safety and performance? (Step 6)

One of the required actions for a CAPA is to update your procedure(s) to reflect any process improvements to eliminate the root cause. When you update procedures, you need to make sure procedural changes do not create a regulatory compliance issue. We do this by inserting a cross-reference to each regulatory requirement in our procedures. The cross-reference is then color-coded and we add a symbol for people that are color blind. Symbols also facilitate electronic searches for regulatory requirements.

If corrective actions you implement involve design changes you will need to repeat design verification and design validation to make sure design changes do not impact safety and performance. If corrective actions change your manufacturing or service processes, you will need to repeat process validation to make sure that the process changes do not impact compliance with your design specifications. These recertification and revalidations steps are frequently forgotten, and they represent the biggest challenge for review and approval of design and process changes (described in the video below).

Perform an effectiveness check – Step 7 of your CAPA procedure

Most people verify CAPA effectiveness by verifying that all the actions planned were completed, but this is not a CAPA effectiveness check. An effectiveness check should use quantitative data from your investigation of the root cause as a benchmark. Then you should verify that the performance after corrective actions is implemented resulted in a decrease in the frequency of the quality problem, a decrease in the severity of the quality problem, or both. Ideally, a process re-validation was performed because validation protocols are required to include quantitative acceptance criteria for success.

Step 8 – Record your CAPA results

You are required to record each step of your CAPA procedure in a CAPA record (FRM-009). Therefore, we created a form that is organized in the order of the CAPA process, and then we wrote the CAPA procedure to match the organization of the form. The biggest mistake we see is that the CAPA owner does not update the record to include all of the details until the CAPA plan is completely implemented. This is a mistake. You should be documenting actions when they are taken. When you gather new information, and you need to update your root cause investigation or your corrective action plan, you are allowed to modify the record. You just need to have a system that allows you to keep track of revisions. This is often referred to as an “audit trail.” If you have a paper-based system, you will need to sign and date the document each time you make an addition. If you revise previous entries, you will need to revise and reprint the CAPA record, and then you will need to sign and date the revised and reprinted CAPA record. Ideally, you will have an electronic system with an audit trail, but software budgets are not infinite.

CAPA procedure step 9 – Close the CAPA record 

The last step in the CAPA procedure is to close your CAPA record. As with most quality system records, the person responsible for the process should review and approve each record for closure with a signature and date. If the person assigned to the CAPA left sections incomplete or made mistakes in completing the CAPA form, the person that made the mistake should be instructed to correct the mistake, identify that they made a correction, and identify the date of the correction. If a CAPA is not effective, then a cross-reference to the new CAPA that is opened should be documented in the older CAPA record.

CAPA Training

If you are interested in more training on CAPA, you might be interested in purchasing Medical Device Academy’s Risk-Based CAPA webinar. 99% of companies hold off on their training until a procedure is officially released as a controlled document. In my experience, however, these procedures seem to have a lot of revisions made immediately after the initial release. New users ask simple questions that identify sections of procedures that are unclear or were written out of sequence. Therefore, you should always conduct at least one training session with users prior to the final review and approval of a procedure. This will ensure that the final procedure is right the first time, and it will give those users some ownership of the new procedure.

After you train your initial group, and after you make the edits they recommended, ask those trainees to review and edit your changes to the procedure. Sometimes we don’t completely understand what someone is describing, and sometimes maybe only half-listening. Going back to those people to verify that you accurately interpreted their feedback is the most important step for ensuring that users accept your new procedure.

After you approve your new CAPA procedure, make sure everyone in your company is trained on the final version of the procedure. CAPA is a critical process (i.e., “the heart”) in your quality system. Everyone should understand it. You should also provide extra CAPA training for department managers, such as root cause analysis training because they will be responsible for implementing CAPAs assigned to their department. You can use this 7-step process for any procedure, but ensure you use it for the most important process of all—your corrective and preventive action process.Spaz helping with the CAPA video 1024x576 CAPA procedure, How do you improve quality and prevent nonconformity?
Thank you for reading to the end of this article. Spaz and I thank you for your support.

Posted in: Uncategorized

Leave a Comment (0) →

Risk management policy – Do you have one?

ISO 14971:2019 includes a requirement for top management to define and document a risk management policy, but do you have one?

Screen capture for POL 005 1024x542 Risk management policy   Do you have one?

Your risk management procedure is not your risk management policy

ISO 14971:2019 includes a requirement for a risk management policy and a risk management procedure. The word procedure is defined (Clause 3.13), a “specified way to carry out an activity or a process,” but there is no definition for policy. Both of these words begin with the letter “p,” but they are not the same.  There is no guidance for a risk management policy in either of the European device regulations for CE Marking and there is no guidance in the US FDA’s regulations. In fact, there is not even a specific cause of the international risk management standard that is specific to the requirement for a risk management policy. The word “policy” only appears in ISO 14971 seven times, but the last occurrence provides the best explanation:

  • Appendix A2.4.2 states that “because [ISO 14971] does not define acceptable risk levels, top management is required to establish a policy on how acceptable risks will be determined.

If someone responsible for risk management activities does not understand this distinction, this shows that risk management training may not be adequate.

Can you have a different policy for each product family?

The purpose of the policy is to establish how the acceptability of risks will be determined. However, not all devices have the same benefit-risk ratio. Therefore, if you have product families with high and low risks, then you should address this in your policy with specific criteria for each device family or create a separate risk management policy for each product family. For example, if your company is focused on designing and developing products for diabetics, you will not have the same benefit-risk profile for a Class 2 glucose reader and lancet for Type 2 diabetics that you have for an automated Class 3 insulin pumps for Type 1 diabetics. In general, separate criteria within one policy are preferred over separate policies to reduce the number of documents that must be managed.

Is there a required format for a risk management policy?

The ISO 14971:2019 standard does not include a specific format or content requirement for your risk management policy. Instead, information about the format and content of a risk management policy is provided in Annex C of ISO/TR 24971:2020. This is a guidance document, and therefore you can choose an alternate approach if you provide a justification for its equivalence. If you choose the approach recommended in Annex C, the following elements should be included:

  • purpose;
  • scope;
  • factors and considerations for determining acceptable risk;
  • approaches to risk control;
  • requirements for approval and review.

You can download Medical Device Academy’s template for a risk management policy (POL-005) by completing the form below.

What are the factors for determining acceptable risk?

There are four possible factors to consider when determining your risk management policy:
  1. Applicable regulatory requirements;
  2. Relevant international standards;
  3. State-of-the-Art;
  4. And stakeholder concerns.

An example of regulatory requirements being applied to the determination of acceptable risks is the special controls defined in 21 CFR 880.5730 for insulin pumps. The special controls requirements outlined by the FDA specify design inputs as well as verification and validation requirements. The requirements are also organized into systems that comprise an insulin pump. For the digital interface requirements, the regulation specifies:

  • secure pairing to external devices;
  • secure data communication between the pump and connected devices;
  • sharing of state information between devices;
  • ensuring the pump continues to operate safely when receiving data that is outside of the boundary limits that are specified as inputs;
  • a detailed process and procedure for sharing pump interface specifications with connected devices.

The hazard implied by the fourth requirement above is that the pump will stop without warning or deliver the incorrect amount of insulin if the data from a continuous glucose sensor is outside of the input specifications. This design input is then addressed by a software design specification established by your company. To verify that your software risk controls are adequate, you will need to execute a verification protocol that automatically inputs a series of values that are outside of the boundary limits specified. Every time a change is made to the software, these boundary limits will need to be re-verified as part of your automated regression analysis to make sure software changes did not have an unintended effect on the device.

For software and use-related hazards, you will not be able to estimate the probability of occurrence of harm. Therefore, you shall assess the acceptability of risks based upon the severity of harm alone. Risk acceptability criteria shall be recorded in your risk management plan and the criteria shall align with your risk management policy. Ideally, these criteria are based upon international standards. For the example of an interoperable insulin pump, the following international standards are applicable:

  • ISO 14971, application of risk management to medical devices
  • IEC 62366-1, application of usability engineering to medical devices
  • IEC 62304, medical device software – software lifecycle processes

For the state-of-the-art, there are three examples provided in the ISO/TR 24971 guidance for how to this relates to your risk management policy:

  1. “Leakage currents of the medical device are state of the art, demonstrated by compliance to the limits and tests regarding leakage current of IEC 60601-1.
  2. Dose accuracy of the delivery device are state of the art, as demonstrated by compliance to the limits and tests regarding dose accuracy of ISO 11608-1.
  3. Protection against mechanical failure caused by impact is on the same level as or better than a similar medical device, as demonstrated by comparative test such as drop test.”

Stake holder concerns is the fourth factor to consider when creating your risk management policy. Stakeholder concerns may be identified in clinical literature. However, the current trend is an emphasis on patient-reported outcome (PRO) data and post-market surveillance. Post-market surveillance is a requirement in ISO 13485, Clause 8.2.1. However, the new European MDR and IVDR have new requirements for post-market surveillance data in the technical documentation. Health Canada updated the medical device regulations to include post-market surveillance summary reports, and even the FDA is trying to develop methods for using real-world data and real-world evidence to make regulatory decisions.

Approaches to risk acceptability

The European device regulations require that a benefit/risk analysis be conducted for all risks and the overall residual risk of your device. The EU regulations also do not permit risk acceptability to consider economic impact. The EU regulations also require that risks are reduced as far as possible. Therefore, if your company is seeking CE Marking, there is only one acceptable approach suggested in ISO/TR 24971, Annex C.2: “reducing risk as far as possible without adversely affecting the benefit-risk ratio.” This is also the approach specified in our risk management procedure (SYS-010).

Requirements for review and approval of the risk policy

Requirements for approval and review of the risk management policy should be specified in the policy itself. This should specify who needs to approve that the policy is acceptable and how often the policy needs to be reviewed. Section 4.2.2 of ISO 14971 also requires that top management review the risk management process for its effectiveness. In general, we recommend that this review of the risk management process be incorporated into the management review process. Therefore, we also believe that this would be the ideal time to review the risk management policy. Generally, this is more frequently than is typically required, but if your risk management process is being reviewed for effectiveness then you have all of the necessary inputs available to review the policy as well.

Posted in: Uncategorized

Leave a Comment (0) →

Formative usability testing – Frequently Asked Questions?

Formative usability testing is not a regulatory requirement, but it is necessary if you want to successfully develop medical devices.

Formative Usability Testing FAQs 1024x169 Formative usability testing   Frequently Asked Questions?

What is the difference between formative and summative usability testing?

“Formative” tests are any usability tests that you perform during the development process, while “summative” testing is the final usability testing you perform to validate that your chosen user interface is effective. Many design teams perform formative testing of one kind or another without even realizing that is what they are doing. Unfortunately, design teams often forget to document the testing they performed during prototyping and product development. Formative usability testing probably always existed as part of product development, but not everyone recognizes the term and identifies the work they have done as “formative.” The most important reason for documenting formative usability testing is to identify which user interface designs failed and why so that future design teams can learn from your failures.

Why don’t more companies do usability testing?

Everyone likes to believe they can skip steps in the learning process, but some lessons can only be learned the hard way. When a medical device design team is developing a user interface for a new product, they need to learn which designs will fail and why before they can fully understand how to design the best user interface for the device. Therefore, most design and development teams will select a user interface that they are familiar with or they see used by a competitor product. The team will not always test the proposed design solution, because they have no reason to believe that the chosen interface will fail. Unfortunately, this can lead to failure later in the design process. Then the team will need to backtrack and repeat the evaluation of various interface designs.

What is the best approach?

“Fail small and fail fast” is the best advice for anyone performing formative usability testing. Instead of writing a lengthy protocol and recruiting 10 subjects to evaluate your proposed user interface, you might consider building a couple of different prototypes and asking two or three people which prototype they prefer and why? Another simple question is, “Tell me what you think of this design?” Iterative formative testing over time with different users is better than one single testing session with a lot of users. It is also better to start collecting formative usability testing data as early in the development process as possible. Gathering data earlier in the process will ensure that users direct the development of your device instead of the design team developing a new device in a direction that is not preferred by users.

When during the design process should formative testing be planned?

Formative testing should be planned during the development phase of the design process. During this phase, medical device manufacturers evaluate multiple design solutions as risk controls for their devices. Use-related risks should be included in this, and the formative usability testing is intended to identify which user interface will do the best job of eliminating the use errors. It is important to evaluate these potential user interfaces and to verify that there are no use errors that the design team overlooked during this phase of the design. This is also the phase of design when the instructions for use are developed and user training is developed. All of this formative usability testing should be completed prior to your design freeze and the start of the verification and validation testing.

What are the different types of formative testing?

Formative usability testing can be used as a pilot for your summative usability testing protocol prior to scheduling the final testing. However, there are many other types of formative testing. The most common reason for doing this testing is to identify any potential use errors that were not originally identified in your user-related risk analysis (URRA). Another type of testing is to simulate use of the device to make sure that every user task is identified in the instructions for use. Finally, design teams will conduct formative usability testing to develop training materials for training new users on how to properly use your medical device.

Which types of formative tests are the most useful?

Use-related risks are difficult to identify unless you conduct simulated use testing with your device. Therefore, you need to get your device in the hands of your intended users, in the intended use environment, and ask them to simulate the use of the device. It is not critical to evaluate a specific number of users. Two or three users might be enough, but simulated use by intended users in the intended use environment is essential to give you the information you need regarding potential use errors. It is also important to avoid “leading” the users. Instead of asking users to perform a specific task, ask users to show you how they would use the device. Ask them what they like about the device, and ask them what they don’t like about the device. Ask users what they think about the device, and ask them how it compares to other devices they are already using.

Who should you recruit for your formative usability testing?

You should start your human factors process by defining the intended user of your device and by defining if there is more than one user group. You then should recruit subjects that are within this user group(s). You can use employees or friends to help you with initial feedback about the usability of your device’s user interface. However, what seems intuitive to one person may be the opposite for other people with different experiences. Even the sequence of steps in which users perform the same tasks can impact usability. Therefore, be very cautious about relying upon data collected only from subjects that are outside your intended user group. Most companies disregard this advice because they are unsure of how to recruit their intended users. However, if your company has difficulty identifying intended users for testing, you will also have difficulty marketing and selling your device later. This struggle may be an indicator that you need to involve marketing and salespeople that can get your prototypes in the hands of the intended users.

How should you document formative studies?

When you are performing summative usability testing you already know exactly what your use-related risks are and you have a list of critical tasks that you are trying to verify users can perform without use errors. Because these tasks are clearly defined, it is easier to write a protocol and it is easier to design data collection forms for study moderators to use. In contrast, when you are conducting formative usability testing you are trying to identify use errors that you are not already aware of. Therefore, it is much harder to write a detailed protocol and design a data collection form. For this reason, it is critical to capture the data with video recordings. This is a safety measure you are taking to ensure that you will not miss valuable use errors or use tasks that you had not already identified. The use of video to record data allows the moderator to focus on observation and interviewing users with open-ended questions. This will generate the most value for your design team during the development process.

Where is testing performed?

While the design team is developing the list of design inputs for your new device, the team must create a definition for the intended users and the intended use environment. The formative usability testing and summative testing should be conducted in the intended use environment or you will need to simulate that use environment. If you are struggling to figure out how to simulate the intended use environment, you should systematically identify the characteristics of the intended use environment. These characteristics include temperature, humidity, ambient noise, other equipment that is present, the number of people present, and the dimensions of the space. If you have a room available with temperature and humidity control, you can add ambient noise by recording the intended use environment. You can rent equipment, or you can place objects of the same size in the space. You can also identify the workspace restrictions by taping the floor to establish boundaries for the simulation. By adding these characteristics to a simulated environment, you open the possibilities for additional places that can be used for formative usability testing.

What will happen if you skip formative testing?

If you skip formative usability testing, you will increase the possibility of failing your summative usability testing. If this happens, then your summative testing becomes your formative usability testing. After you fail, you will need to revise your testing protocol and repeat the study. Another possibility is that you will fail to identify a potential use error. If the FDA identifies this use error you will need to repeat your testing. If the use error is never identified, then you may end up with complaints or medical device reporting of use errors. In extreme cases, this could result in serious injuries or death.

Posted in: Uncategorized

Leave a Comment (0) →

Is monitoring every procedure required?

Process monitoring is required but do you know whether monitoring every procedure is required by the FDA QSR or ISO 13485?

One of the elements that Medical Device Academy has incorporated into each procedure we created in our turnkey quality system is a section titled, “monitoring and measurement.” The purpose of this section is to force each process owner to identify a process metric for monitoring every procedure. In some cases, we suggest a metric that would be appropriate for most companies establishing a new quality system. In other procedures, we use the following default text:

Enter a quality metric that you want to track for this process in accordance with ISO 13485:2016, Clause 8.2.5 and the procedure for Monitoring, Measurement, and Analysis (SYS-017).

Where are the requirements for process monitoring in 21 CFR 820?

Some of the companies that have purchased our turnkey quality system have asked, “Is it required to monitor and measure something in every procedure?” In general, it is not a specific requirement to have a metric specified in each procedure. In fact, if your quality system is not ISO 13485 certified, there are actually only a few places where the US FDA requires monitoring. The FDA does not have a section specific to monitoring and measurement of processes, but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used:

  • 21 CFR 820.100(a)(1) – “Analyzing processes, work operations, concessions, quality audit reports, quality records, service records, complaints, returned product, and other sources of quality data to identify existing and potential causes of nonconforming product, or other quality problems. Appropriate statistical methodology shall be employed where necessary to detect recurring quality problems;”
  • 21 CFR 820.200(b) – “Each manufacturer shall analyze service reports with appropriate statistical methodology in accordance with § 820.100.”
  • 21 CFR 820.250 – “(a) Where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics. (b) Sampling plans, when used, shall be written and based on a valid statistical rationale. Each manufacturer shall establish and maintain procedures to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed. These activities shall be documented.” Note: the other two instances are the title of 21 CFR 820.250.

The word “monitoring” is equally rare (i.e. 4x) in the QSR:

  • 21 CFR 820.70(a) – “Each manufacturer shall develop, conduct, control, and monitor production processes to ensure that a device conforms to its specifications…Where process controls are needed…(2) Monitoring and control of process parameters and component and device characteristics during production.”
  • 21 CFR 820.75(b) – “Each manufacturer shall establish and maintain procedures for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met…(2) For validated processes, the monitoring and control methods and data, the date performed, and, where appropriate, the individual(s) performing the process or the major equipment used shall be documented.”

Where are the requirements for process monitoring in ISO 13485:2016?

ISO 13485:2016 has a section specific to monitoring and measurement of processes (i.e. Clause 8.2.5). In addition, the word “monitoring” occurs 52 times in the standard and there are 60 incidents of some variant or the exact word. , but there is a section of the regulations specific to statistical techniques (i.e. 21 CFR 820.250). However, it does not state in the QSR that statistical analysis is required for all processes. In fact, there are only six instances where the word “statistical” is used. There are four Clause headings that actually include the word monitoring:

  • Clause 7.6, Control of monitoring and measuring equipment
  • Clause 8.2, Monitoring and measurement
  • Clause 8.2.5, Monitoring and measurement of processes
  • Clause 8.2.6, Monitoring and measurement of product

In Clause 1, Scope, and Clause 4.1.5, the Standard states that any outsourced processes remain the responsibility of the company and must be accounted for in the quality system by monitoring, maintaining, and controlling the processes.

Monitoring of risk is included in the definition of “risk management” in the Standard (i.e. Clause 3.18).

Clause 4.1.3 states that the organization shall, “b) ensure the availability of resources and information necessary to support the operation and monitoring of these processes…d) monitor, measure as appropriate, and analyze these processes.”

Clause 4.2.3 states that the contents of the Medical Device File (i.e. MDR or TF), shall include, “d) procedures for measuring and monitoring.”

Monitoring and measurement of processes and product are required inputs to the Management Review in Clauses 5.6.2e) and f).

Clause 6.4.1 requires a procedure for monitoring the work environment if it can have an effect on product quality.

Clause 7.1 requires the company to consider including monitoring in product realization planning.

Clause 7.4.1 requires a plan for monitoring suppliers.

Clause 7.5.1 requires monitoring production and service, including the monitoring of process parameters and product characteristics.

Clause 7.5.6 requires monitoring of validated process parameters.

Clause 7.5.8 requires identification of status with regard to product monitoring and measurement (i.e. inspection status).

Clause 7.6 requires monitoring and measurement of calibrated devices and validation of any computer software used to monitor calibrated devices.

Clause 8.1 states that companies shall plan and implement monitoring and measurement of processes.

Clause 8.2 is titled, “Monitoring and measurement.”

Clause 8.2.1 requires monitoring of customer feedback.

Clause 8.2.5 requires monitoring of processes to ensure planned results are achieved.

Clause 8.2.6 requires monitoring of products to ensure product requirements have been met.

Clause 8.4 requires data analysis of monitoring data from at least six different processes:

  1. Feedback
  2. Conformity to product requirements
  3. Characteristics and trends of processes and products, including opportunities for improvement
  4. Suppliers
  5. Audits
  6. Service reports, as appropriate

In summary, while not every single clause that requires a procedure includes a requirement for monitoring, there are a number of processes where the requirement to monitor the process is explicitly stated.

Why do all of our procedures include the requirement for metrics?

Medical Device Academy expanded the requirement for monitoring to all procedures for five reasons:

  1. Quality objectives must be “established at relevant functions and levels within the organization.” Therefore, establishing monitoring requirements for each procedure ensures that top management has metrics for every process and a lack of data is never an excuse for not establishing a new quality objective when improvement is needed.
  2. If every procedure has a requirement for monitoring, then employees don’t have to remember which processes require monitoring and which processes do not explicitly require monitoring.
  3. The process approach to auditing includes metrics of the process as one of the seven items that are included in every process turtle diagram, and therefore, including metrics for each procedure facilitates the process approach to auditing.
  4. If a company does not have a process metric already established, it is often difficult to perform an investigation of the root cause of quality issues. If a metric is already being monitored for the process, this facilitates the investigation of the root cause and you can use the baseline monitoring data to help you establish effectiveness criteria for the corrective action.quantitative effectiveness check 300x209 Is monitoring every procedure required?
  5. Finally, most companies struggle to identify preventive actions as required by Clause 8.5.3, and we have found that data analysis of monitoring data is the best source of identifying new preventive actions.

What are the disadvantages when you monitor and measure something in every procedure?

The primary reason for resistance to identifying a metric for monitoring in every procedure is that it will increase the workload for the employees responsible for that process. However, monitoring of data does not always increase workload. In fact, when process data is recorded in real-time on a run chart it is often possible to identify a trend much earlier than when data is simply recorded and subjected to monitoring.

  • Example #1: The automatic tracking of toner in a printer tells HP when to ship you a new toner cartridge before you need it. This ensures that there is no loss in productivity because you never run out of ink or the ability to print documents.
  • Example #2: Companies will use project management software (e.g. Asana) to monitor labor utilization. This will help identify when a specific resource is nearing capacity. When this occurs, the project manager can add time buffers to prerequisite steps and adjust the starting date of the resource-limited tasks to an earlier starting date. This ensures that more time is available to finish the task or to take advantage of resource availability at an earlier date.
  • Example #3: Monitoring the revision date for procedures helps the document control process owner identify procedures that should be evaluated for the need to be revised and updated. Often this is articulated as a quality objective of reviewing and updating all procedures within 2 years. This also ensures that procedures remain current and compliant with regulatory requirements.

What are the advantages of monitoring every procedure?

The phrase “what gets measured gets managed” is a popular business philosophy that implies measuring employee activity increases the likelihood that employees will complete a task or perform it well. In contrast, if a process is not monitored, employees may assume that it is not important and the tasks may be skipped or completely forgotten. Setting quantitative goals is also sometimes integrated with economic incentives or bonuses that are granted to individuals and teams.

FDA transition from QSR to ISO 13485

The US FDA is planning its transition from 21 CFR 820 to ISO 13485 as the quality system criteria. This will force companies to make adjustments to their quality systems and increase the amount of process monitoring performed. My general advice is to work with employees that are performing tasks to identify streamlined methods for monitoring those tasks without being overly burdensome. Then you and the employees you manage can analyze the data together and identify opportunities for improvement. When you do this, experiment with manual methods using whiteboards and paper charts that are visible in public areas first. Only implement automated solutions after you have optimized the data being collected and the frequency of data collection, and remember that not every process will benefit from automated statistical process control. Sometimes the simple approach is best.

Posted in: Uncategorized

Leave a Comment (0) →

Individual process audits or one full quality system audit, which is better?

 

You can conduct multiple individual process audits or you can conduct one full quality system audit, but which solution is better?

What are individual process audits?

There are 25 processes that require procedures for compliance with the US FDA quality system regulations and ISO 13485:2016 has 28 required procedures. Individual process audits focus on one of these procedures, the process it controls, the equipment and software used by that process, the work environment where the process is performed, the people responsible for the process, the records resulting from that process, and any metrics or quality objectives associated with that process. An individual process audit can be completed in remotely or on-site, and these audits will be much shorter in duration than a full quality system audit. Another way to think of an individual process audit is to realize that a full quality system audit is comprised of many individual process audits scheduled back-to-back. Auditing one process might be as short in duration as 30 minutes (e.g. control of records) but individual process audits can take as long as four hours (e.g. design controls and technical file audits).

What is a full quality system audit?

A full quality system audit is typically a single audit conducted annually to address all the requirements for conducting an internal audit of your quality system. In this type of audit, all of the procedures and processes should be covered. Therefore, full quality system audits are necessarily longer. If the person assigned to conduct the full quality system audit is an employee, that person cannot audit their own work. This can be addressed in two ways: 1) the audit can be a team audit, and the other team members can audit areas the lead auditor was responsible for; and 2) the process(es) that the lead auditor is responsible for can be audited as individual process audits by another auditor at another time.

If the person assigned to conduct the full quality system audit is a consultant from outside the company, there is still potential for conflicts regarding independence. If the consultant audited the company in the previous year, then the auditor cannot audit last year’s internal audit. In our consulting firm we address this issue in two ways: 1) we rotate who is assigned to audits so that the same auditor does not conduct a full quality system audit two years in a row, or 2) we assign another auditor in our company to conduct the audit of internal auditing as a team member.

How do you evaluate auditing effectiveness?

Some companies perceive that auditing is a necessary evil and they want to put as little effort and resources into the audit as possible. In this situation, auditing might be evaluated based upon whether it was completed on-time, by how much the audit cost the company, and the fewer nonconformities identified the better the perceived outcome. This perspective typically results in a single full quality system audit that is three days in duration or shorter if an auditor can manage to complete the audit in less time. Of course the shorter the audit is, the fewer records that an auditor has time to review. Therefore, shorter audits typically have fewer findings and management is pleased at the outcome because the audit required fewer resources and had little or no nonconformities.

The better approach is to look at auditing as a method for identifying areas that need improvement. Identifying areas where your quality system needs improvement is the intent of requiring internal audits. Therefore, the amount of time your company allocates to auditing should reflect the benefits for improvement that are identified. Top management of your company needs to identify which process areas they feel needs improvement. Only then can the audit program manager design an audit schedule that will focus on identifying opportunities for improvement and nonconformities in the process areas where management feels improvement is most needed. Ideally, this approach to auditing will focus on looking for inefficiency and metrics with negative trends. These findings result in preventive actions instead of corrective actions, because the process is not yet nonconforming. In general, the more opportunities for CAPAs that are identified the more valuable the audit was.

What advantages do one full quality system audit present?

Sometimes a single full quality system audit is easier to schedule, because it is only once per year. The rest of the year your company will not need to spend much time discussing audits or even thinking about them. If your company perceives audits as a necessary evil, then the less disruption caused by scheduling an audit the better.

Another advantage of conducting full quality system audits is that you can more easily afford to use external consultant auditors, because the travel costs for auditing are limited to one trip per year. If you had more than twenty individual process audits each year, and external consultant auditors conducted all of the audits, then you would have to pay for travel costs twenty times each year. Unless the consultant lives locally, these travel costs can be substantial.

What advantages exist for individual process audits?

Individual process audits are much easier for the auditor to complete within the time established in the audit agenda, because the auditor does not have another audit process immediately proceeding or immediately after the process they are auditing. There are also fewer people that need to attend an opening or closing meeting for an individual process audit, because only one process is being audited. Managers from other departments are seldom needed for participation in the opening or closing meeting.  The combined benefits result in the auditor being more likely to start the opening meeting on-time and to start the closing meeting on-time.

The shorter duration of individual process audits is also an advantage. There are very few times in a year when none of your department managers will be traveling, sick, or on vacation. These rare weeks only happen a few times each year, and sometimes auditors must proceed with an audit even if someone is absent because they have no alternative. If you are preparing for an audit remotely, you face-to-face audit time is only 90 minutes, and your report writing time is also conducted remotely, then finding 90-minutes of available time in an department manager’s schedule is usually quite easy.

Can both approaches to internal audit scheduling coexist?

You can combine both approaches to audit scheduling in several possible ways. First you can schedule one full quality system audit each year in order to make sure that the minimum audit requirements are met, and then top management can review the results of the full quality system audit to decide which processes would benefit from individual process audits.

A second strategy would include conducting individual process audits for each process that resulted in a nonconformity during 3rd party certification audits or during the one full quality system audit. In this scenario, you might have a 3rd party audit in November, a full quality system audit in May, and top management might select 10 other individual processes to audit during the other 10 months of the year.

A third strategy would be to alternate between individual process audits and single full quality system audits each year. During “odd” years the audit program manager would only schedule one full quality system audit, and during “even” years the audit program manager would schedule multiple individual process audits.

A fourth strategy would be for top management to select a few processes that they would like the audit program manager to focus on with individual process audits, and all of the remaining processes would be incorporated into a single audit that covers the remaining 70% of the quality system.

Each of these four strategies for combining the two approaches to audit scheduling is viable and may result in multiple opportunities for improvement being identified. There is no regulation that favors one approach over another, but all four strategies require more time an effort on the part of the audit program manager and top management to discuss and plan the annual audit schedule.

Next steps if you would like to try individual process audits

If your company has always scheduled a single full quality system audit each year, you can test the concept of conducting an individual process audit by selecting just one process to audit. The best choice for this approach is to pick a process that has one or more CAPAs that are in progress or to select a process that top management feels is performing efficiently. The more frustration that top management experiences with a process, the greater the need is to identify opportunities for improvement. If the company has not already identified CAPAs to initiate for that process, you might just need an outsider to state the obvious: “I think we need a CAPA in this department.” The outsider might be a consultant, but it could also be a person from another department. If you would like a quote for an individual process audit, please visit our audit quote webpage.

About the Author

Rob Packard 150x150 Individual process audits or one full quality system audit, which is better?

Rob Packard is a regulatory consultant with 25+ years of experience in the medical device, pharmaceutical, and biotechnology industries. He is a graduate of UConn in Chemical Engineering. Robert was a senior manager at several medical device companies—including the President/CEO of a laparoscopic imaging company. His Quality Management System expertise covers all aspects of developing, training, implementing, and maintaining ISO 13485 and ISO 14971 certification. From 2009-2012, he was a lead auditor and instructor for one of the largest Notified Bodies. Robert’s specialty is regulatory submissions for high-risk medical devices, such as implants and drug/device combination products for CE marking applications, Canadian medical device applications, and 510(k) submissions. The most favorite part of his job is training others. He can be reached via phone 802.258.1881 or email. You can also follow him on Google+LinkedIn or Twitter.

Posted in: Uncategorized

Leave a Comment (1) →