In this formative usability testing webinar, you will learn how and why to perform formative usability testing.
Formative Usability Testing Webinar ($79)
Please note: This product will be delivered to the email address provided in the shopping cart transaction. After verifying the transaction, please check your email for the download (including your spam folder).
When is the Formative Usability Testing Webinar?
This webinar will be recorded in sections and will not be available live. The final version will be released on Monday, March 11, 2024. You can purchase it on-demand and watch the training as often as you wish.
What you will receive
- A Formative Usability Testing Protocol Template
- Access to download the recording of the webinar from our Dropbox folder
- Native slide deck for this webinar
- FDA Guidance on Human Factors
There are 24 slides in the presentation slide deck, and the presentation is 62 minutes in duration. All content deliveries will be sent via AWeber emails to confirmed subscribers. Below are the top ten frequently asked questions that we receive about formative usability testing. These frequently asked questions and the answers provided are examples of the information you will learn in our formative usability testing webinar.
What is the difference between formative and summative usability testing?
“Formative” tests are any usability tests you perform during development. At the same time, “summative” testing is the final usability testing you perform to validate that your chosen user interface is effective. Many design teams perform formative testing of one kind or another without even realizing that is what they are doing. Unfortunately, design teams often forget to document the testing they performed during prototyping and product development. Formative usability testing probably always existed as part of product development, but not everyone recognizes the term and identifies their work as “formative.” The most important reason for documenting formative usability testing is to identify which user interface designs failed and why so that future design teams can learn from your failures.
Why don’t more companies do usability testing?
Everyone likes to believe they can skip steps in the learning process, but some lessons can only be learned the hard way. When a medical device design team is developing a user interface for a new product, they need to learn which designs will fail and why before they can fully understand how to design the best user interface for the device. Therefore, most design and development teams will select a user interface they are familiar with or see used by a competitor product. The team will not always test the proposed design solution because they have no reason to believe the chosen interface will fail. Unfortunately, this can lead to failure later in the design process. Then the team will need to backtrack and repeat the evaluation of various interface designs.
What is the best approach?
“Fail small and fail fast” is the best advice for anyone performing formative usability testing. Instead of writing a lengthy protocol and recruiting ten subjects to evaluate your proposed user interface, you might consider building a couple of different prototypes and asking two or three people which prototype they prefer and why. Another simple question is, “Tell me what you think of this design?” Iterative formative testing over time with different users is better than one testing session with many users. It is also better to start collecting formative usability testing data as early in development as possible. Gathering data earlier in the process will ensure that users direct the development of your device instead of the design team developing a new device in a direction that users do not prefer.
When should formative testing be planned during the design process?
Formative testing should be planned during the development phase of the design process. During this phase, medical device manufacturers evaluate multiple design solutions as risk controls for their devices. Use-related risks should be included in this, and the formative usability testing is intended to identify which user interface will do the best job of eliminating the use errors. It is important to evaluate these potential user interfaces and verify that there are no use errors that the design team overlooked during this design phase. This is also the phase of design when the instructions for use are developed and user training is developed. This formative usability testing should be completed before your design freeze and the verification and validation testing start.
What are the different types of formative testing?
Formative usability testing can be used as a pilot for your summative usability testing protocol before scheduling the final testing. However, there are many other types of formative testing. The most common reason for doing this testing is to identify potential use errors not originally identified in your user-related risk analysis (URRA). Another type of testing is to simulate the device’s use to ensure that every user task is identified in the instructions for use. Finally, design teams will conduct formative usability testing to develop training materials for training new users on properly using their medical device.
Which types of formative tests are the most useful?
Use-related risks are difficult to identify unless you conduct simulated use testing with your device. Therefore, you need to get your device in the hands of your intended users, in the intended use environment, and ask them to simulate the use of the device. It is not critical to evaluate a specific number of users. Two or three users might be enough, but simulated use by intended users in the intended use environment is essential to give you the information you need regarding potential use errors. It is also important to avoid “leading” the users. Instead of asking users to perform a specific task, ask users to show you how they would use the device. Ask them what they like about the device and what they don’t like about it. Ask users what they think about the device and how it compares to other devices they are already using.
Who should you recruit for your formative usability testing?
You should start your human factors process by defining the intended user of your device and by defining if there is more than one user group. You should recruit subjects within this user group(s). You can use employees or friends to help you with initial feedback about the usability of your device’s user interface. However, what seems intuitive to one person may be the opposite for others with different experiences. Even the sequence of steps where users perform the same tasks can impact usability. Therefore, be cautious about relying upon data collected only from subjects outside your intended user group. Most companies disregard this advice because they are unsure how to recruit their intended users. However, if your company has difficulty identifying intended users for testing, you will also have difficulty marketing and selling your device later. This struggle may be an indicator that you need to involve marketing and salespeople that can get your prototypes in the hands of the intended users.
How should you document formative studies?
When performing summative usability testing, you already know your use-related risks. You have a list of critical tasks you are trying to verify that users can perform without errors. Because these tasks are clearly defined, writing a protocol and designing data collection forms for study moderators to use is easier. In contrast, when conducting formative usability testing, you are trying to identify use errors that you are not already aware of. Therefore, writing a detailed protocol and designing a data collection form is much harder.
For this reason, it is critical to capture the data with video recordings. This is a safety measure to ensure that you will not miss valuable use errors or tasks you have not already identified. Using video to record data allows the moderator to focus on observation and interviewing users with open-ended questions. This will generate the most value for your design team during development.
Where is testing performed?
While the design team is developing the list of design inputs for your new device, the team must create a definition for the intended users and the intended use environment. The formative usability and summative testing should be conducted in the intended use environment or you will need to simulate that use environment. If you are struggling to figure out how to simulate the intended use environment, you should systematically identify the characteristics of the intended use environment. These characteristics include temperature, humidity, ambient noise, other equipment, the number of people present, and the dimensions of the space. If you have a room with temperature and humidity control, you can add ambient noise by recording the intended use environment. You can rent equipment or place objects of the same size in the space. You can also identify the workspace restrictions by taping the floor to establish boundaries for the simulation. Adding these characteristics to a simulated environment opens the possibilities for additional places that can be used for formative usability testing.
What will happen if you skip formative testing?
If you skip formative usability testing, you will increase the possibility of failing your summative usability testing. If this happens, then your summative testing becomes your formative usability testing. After you fail, you must revise your testing protocol and repeat the study. Another possibility is that you will fail to identify a potential use error. If the FDA identifies this use error, you must repeat your testing. If the use error is never identified, you may have complaints or medical device reporting of use errors. In extreme cases, this could result in serious injuries or death.
About Your Instructor
Rob Packard is a regulatory consultant with ~25 years of experience in the medical device, pharmaceutical, and biotechnology industries. He is a graduate of UConn in Chemical Engineering. Rob was a senior manager at several medical device companies—including the President/CEO of a laparoscopic imaging company. His Quality Management System expertise covers all aspects of developing, training, implementing, and maintaining ISO 13485 and ISO 14971 certifications. From 2009 to 2012, he was a lead auditor and instructor for one of the largest Notified Bodies. Rob’s specialty is regulatory submissions for high-risk medical devices, such as implants and drug/device combination products for CE marking applications, Canadian medical device applications, and 510(k) submissions. The most favorite part of his job is training others. He can be reached via phone at 802.281.4381 or by email. You can also follow him on YouTube, LinkedIn, or Twitter.