Amazon’s Alexa Voice Service in Healthcare

Macadamian Technologies | June 29, 2016 | 5 Min Read

The Amazon Echo and Alexa voice service have been a hit in the consumer market with skills that help you order a pizza or call an Uber. Now with the introduction of the "KidsMD" skill, the Alexa voice service is making its way into the healthcare market. Having focused on healthcare IT and stressing the possibilities of voice interaction for years, it’s great to see Alexa for healthcare finally in action.

At Macadamian we’ve been doing a lot of work with Alexa, Amazon’s voice interaction system. Alexa lets you ask questions like “Alexa, how many days left until Christmas?” and “Alexa, can you order me a pizza?”

Now, thanks to new software developed by Boston Children’s Hospital, users will be able to inquire if their child’s fever is serious enough that they should see a doctor. “KidsMD”, the skill from Boston Children’s Hospital, is among more than 1000 skills currently available for the Echo and one of the first in the medical space. Having focused on healthcare IT and stressing the possibilities of voice interaction for years, it’s great to see Alexa for healthcare finally in action.

Why Alexa for Healthcare?

Alexa’s voice recognition capabilities have received many positive reviews, and the interaction is simple and easy. Amazon is pushing for widespread adoption of Alexa with a $100M venture fund that encourages the development of new skills or physical devices leveraging Alexa, and companies making contributions to voice technology. With all of this activity, it will not be long before Alexa becomes a commonplace way of interacting with everything from retail outlets to banks and school. Doctors, nurses, pharmacists, as well as patients and their caregivers, will soon all be familiar with the Alexa paradigm from use in their personal lives.

Alexa is hands-free, meaning you can activate it even if your hands are injured, dirty, or just busy. You can interact with it even in a noisy environment like a hospital admissions area. It has the potential of enabling continuity of care, as patients can interact with it the same way whether in a hospital setting or at home.

Potential Uses of Alexa in Healthcare IT

All the usefulness of a patient portal, without the hassle of logging into your computer:

  • Answering a patient’s questions about their condition, symptoms, medications, and upcoming appointments
  • Request medication refills or book an appointment
  • Provide updates on vitals to a remote nurse

Voice recognition for specialized patient uses:

  • Help patients with poor eyesight or limited mobility (as the accessibility of traditional healthcare software is notoriously poor for these groups)
  • Bed-ridden patients, whether at home or in the hospital, could use Alexa to control the room temperature or blinds, control their TV’s, music, even order an Uber to get to an appointment.

As an admissions kiosk in the hospital or ER:
One of the teams at this year’s MacHack actually developed a demo of how this could work.

For clinicians on the job:

  • Ask Alexa for information on a particular drug or procedure. For example, one could imagine Elsevier developing an Alexa skill for nurses to get answers from the Clinical Key reference material, or CPhA developing an Alexa skill that allows you to ask for pharmaceutical information from their online database
  • Ask for help on diagnosis and treatment (clinical decision support)
  • Call other hospital workers for assistance
  • Operate software or medical devices remotely while hands are dirty
  • Record key information within a patient encounter, for later transcription
  • Request a patient discharge


Creating an Intuitive Voice Interaction Experience

We’ve spent the last decade talking about creating intuitive, user-friendly, visually appealing UI screens. But suddenly with Alexa, screens have disappeared, the experience is now voice-only! If you’ve only done wireframe ‘screen’ design, creating an intuitive voice experience may seem challenging. While Alexa is strong on general voice recognition, your skill will only be as good as the voice interaction experience you provide.

The voice experience must:

  • Take into account the context of the user (where are they, what are they trying to accomplish, what is the topic at hand, etc.).
  • Offer valuable functionality at the right time (what are the questions and services users of your system really need in an Alexa context?)
  • Anticipate the types of questions users will ask and how they will be worded
  • Use the right natural language and terminology to have a natural, domain-appropriate conversation (you might say ‘pink eye’, a doctor might say ‘conjunctivitis’)

Luckily designing an Alexa voice experience for your skill is not something you have to completely learn from scratch. The principles of user-centered design that are core to traditional computer screen workflow design still apply in the voice interaction world.

  • Work with a UX research professional and/or domain experts to understand the domain and context, the user’s mental models, and what features will be most valuable in an Alexa context
  • Clearly defining the ‘persona’ or personae who will use the system (e.g. cardiac patients who have just returned home after a hospital stay and have to work through a care plan)
  • Design the skill using established voice interaction design patterns and best practices
  • Employ usability testing to validate the user interaction is as smooth as you hoped, and highlight stumbling areas that need to be fixed before release
  • Test and iterate frequently, until the level of quality in the voice experience is there

Limitless potential, but you need to know Alexa

Like any new technology, Alexa has its quirks. There are limitations of the technology today, and areas for improvement that developers today need to work around to create a healthcare skill for early adoption, like the KidsMD skill.

Some of these limitations to be aware of include:

Unseen errors

Often a word can be misheard by the system and a different (but correctly spelled) word can be used in its place. These “typos” are often hard to detect as Amazon doesn’t provide visual feedback and could pose a risk to patient safety.

Mapping different ways of saying the same thing

Alexa allows you to create terminology ‘mappings’ to ensure users can speak casually rather than have to use a rigid set of predetermined words, and it will interpret correctly.

Domain-specific cases

The Alexa sample utterances may need to use different spellings to make the recognition work. Also, the grammar-based language model definition of Alexa should provide an understanding of domain-specific medical terms that would be more reliable than what Siri and Google Assistant can provide.

Context-specific intents

Use thesession.attributes functionality of the Skill Kit to make sure Alexa remembers the context of previous requests, so users can build on the conversation with follow-up questions.

Cognitive Load

The biggest challenge is the “what can I say?” aspect for a user, in that they need to remember the invocation and action phrases. If a skill can’t be built to query the user (the current limitation), to be truly useful it may require other user support in the form of a cheat-sheet or a very good implementation of “Alexa, what can I say?”

Webinar: Applications of Voice Assistants in Healthcare

Think voice assistants are still off in the future? In this webinar we detail practical real world applications of voice assistants in healthcare.

Suggested Stories

Applications of Voice Assistants in Healthcare

Discover how organizations across the continuum of care can leverage the growing consumer demand for voice-enabled devices to achieve an extensive list of objectives from increased patient engagement to improved outcomes and lowered care costs.

Read More

Voice UI Design Best Practices

Voice assistants are poised to transform customer engagement as well as business models. Discover why voice is the next digital frontier – and what you should know about voice-first solutions.

Read More

Structuring Multidisciplinary Software Teams

5 strategies we've learned from working with the biggest names in software for structuring multidisciplinary software teams to get amazing software out the door fast.

Read More
Macadamian has been acquired by Emids 🎉
This is default text for notification bar