Siemens Medical Solutions USA, Inc. Ultrasound Business Area made a concerted effort to improve the overall usability of their premium ultrasound systems, the new ACUSON S Family™ of ultrasound systems, HELX™ Evolution with Touch Control. The systems utilize advanced technologies and diagnostic tools to expand clinical capabilities. Siemens felt confident that they had achieved their goals and wanted to measure users’ performance and determine user satisfaction with one of these systems, the ACUSON S3000™ ultrasound system. In order to accomplish this, Siemens needed to be able to measure the outcomes of their efforts in an objective way.
Macadamian’s 10+ years of experience with usability testing and knowledge in the healthcare domain made the firm an excellent candidate for the job. “After talking with many user experience companies, we felt Macadamian had the expertise to deliver on our requirements,” says Doug Younger III, Global Director of Marketing, General Imaging/ Radiology Segment, Siemens Healthcare Ultrasound. The pace at which projects are delivered and the quality of work contributed to Macadamian being selected to work on this project.
Ultrasound systems should support a process that provides a high level of usability for all users. At the request of Siemens, user experience researchers at Macadamian evaluated the usability of the Siemens system by conducting a comprehensive usability test.
The usability test was conducted in a third-party usability research lab set up to resemble a clinical environment for sonographers. After a careful screening process, 20 sonographers were recruited to act as test participants. Participants had between 5 – 40 years of experience and regularly conduct abdominal ultrasound exams. None of the participants had prior experience with the Siemens system.
Macadamian researchers directed, filmed, and edited a training video that participants watched prior to the usability test. This video ensured each participant received consistent training. After watching the video, each participant was given the same amount of time to become familiar with the system.
Following the training, they completed 11 common tasks in abdominal sonography. Macadamian researchers worked with expert sonographers to formulate these representative tasks.
During the usability test, the researchers captured metrics for effectiveness, efficiency, and user satisfaction. To assess the effectiveness of the system, task success and error rates were collected and the number of deviations measured efficiency of use. To assess the satisfaction of the system, Macadamian collected ease of use ratings after each task and qualitative feedback.
Additionally, at the end of the usability session, participants were asked to complete the System Usability Scale questionnaire. The questionnaire is standard practice in usability testing where the participants are asked to rate various aspects of the system on its usability and learnability.
Siemens found the results of the study favorable and the data collected will empower the Siemens Ultrasound Sales Team when reaching out to potential customers. For the Siemens Ultrasound Human Factors Engineering team, the usability test identified the areas that would help improve the interactive experience with the system and led to the discovery of design recommendations for future releases. Participants also identified additional aspects of the ultrasound system that were out of scope for this study which informed the team of areas to evaluate in the future (e.g. image quality).
Macadamian’s success with this research is leading towards similar usability studies for the Siemens’ Ultrasound Division as well as other healthcare divisions within Siemens. The data resulting from this research will also be used to support future research in the Ultrasound Division.
Webinar – 6 Usability Testing Best Practices With Siemens Healthineers
View an introduction to the webinar above or watch the full webinar.
To see the results of the study and for more information, download the full case study below.