The FDA’s position on validation testing requires that testing be conducted in actual or simulated environments. This focus has put the spotlight on the benefits of medical simulation centers. These centers have the ability to recreate sophisticated hospital environments and enable complex medical tasks to be performed in true-to-life scenarios, allowing manufacturers to determine whether a device is safe and effective before exercising it in a real clinical setting such as a clinical trial. Farm uses these facilities when appropriate to conduct summative testing for our clients.
Typically established as training centers for medical professionals, medical simulation centers are resource-rich environments that can be transformed into many types of hospital settings, such as operating rooms with a scrub-in area, labor and delivery suites, or acute care settings. We’ve found simulation centers typically have access to most of the necessary supplies and equipment used in an actual hospital setting, such as surgical gloves and dressings, crash or drug carts, syringes, IV poles, infusion pumps, ultrasound machines, and patient monitors—you name it. Specialized equipment, as well as the device being tested, can also be brought in.
Most medical simulation centers give researchers the ability to mock up the conditions of device use. For example, one can alter the height of the patient bed; enable alarms, overhead pages, and other medical device sounds; and vary lighting to match typical use conditions. The staff members who manage the simulation centers often have medical backgrounds and can provide their expertise and assistance in setting up realistic scenarios.
Simulation centers vary in their level of fidelity (or realism). When planning a usability validation test, researchers should consider the level of fidelity required based on the type of device, the environment in which it is used, and the level of impact environmental conditions could have on user interactions with the device. For example, it may be important to test a device designed for use in an emergency room setting in a high-fidelity simulated environment (such as the Center for Medical Simulation in Cambridge MA) in order to consider how frequent distractions influence the use of the device. According to ANSI/AAMI’s HE75:2009 guidance document, “ER staff, in particular, are regularly interrupted because of the unpredictable nature of their work environment…In one study, ER physicians were three times more likely to be interrupted than their primary-care peers working in medical offices and spent more time managing multiple patients simultaneously than primary-care physicians.”i High-fidelity labs may have more capabilities to recreate ER-like auditory distractions that typically occur, such as alarms, fans, and other sounds from the presence of multiple medical devices, staff and patient conversations, intercom pages, physical commotion, etc. In contrast, low-fidelity labs may be sufficient in cases where the physical and/or clinical environment is much less complex and much more controlled.
Since the primary focus of medical simulation centers is to educate medical professionals, many are equipped with high-fidelity video cameras and audio equipment. Students are recorded performing various procedures and the video is used as a teaching mechanism to improve their skills. Faculty can also use these videos to evaluate the abilities and techniques of students. For usability validation testing purposes, the audio and video capabilities allow stakeholders to observe sessions remotely or from a separate room and provide a record of each test session that may be reviewed later during data analysis.
In some cases staff members or actors (cohorts) play the role of patients, medical professionals, or relatives. These actors can be tasked with introducing realistic interruptions, adding stress or pressure to the medical scenario at hand. For example, they may act highly emotional, ask difficult questions, or interfere with a physician during an important step or procedure.
Mannequins are often used to stand in for patients during a usability test. High-end mannequins such as METI man and Blue Phantom trainers offer a plethora of capabilities, including adjustable internal bleeding levels, tissue that matches the real acoustic characteristics of human tissue and can be used with ultrasound, sensors that can detect the depth of nasal or oral intubation tubes, chest rise and lung sounds that can be synchronized with different breathing patterns, and pulse strength and blood pressure that can vary depending on ECG readings. We’ve seen adult female mannequins that can simulate childbirth and neonates that can produce various types of cries. The high-fidelity anatomy of these simulators, along with the ability to make them “speak” to medical professionals (a function operated by staff members from a control room), translates into an experience that very closely mimics genuine hospital scenarios.
In this ABC News video students at New York’s Simulation Center receive life-like lessons from high-tech mannequins.
As prescribed in the international standard IEC 62366:2007, “Usability validation may be performed in a laboratory setting, in a simulated use environment, or in the actual use environment.”ii Given all of their capabilities, medical simulation centers are worth considering for medical device validation tests. Be warned, however, that it isn't always the right solution for medical device usability testing. Their usefulness is a matter of applicability. Researchers must also consider whether patient behavior would significantly affect the outcome of the test, because if so, it may be necessary to evaluate the device in an actual clinical setting using real human patients.
Personally, I’m amazed at what one can accomplish in the simulation centers and eager to see what the next wave of technology innovation may bring!
References
i Association for Advancement of Medical Instrumentation, ANSI/AAMI HE75:2009
ii IEC/ISO International Standard 62366, Edition 1.0, 2007-10
