Design Conversational AI with These 5 Expert Tips in Mind

Kristen Stephens

There are a few key ingredients that go into crafting powerful, accurate, and fast voice experiences—including branded wake words, good user data, defined voice personas, and multiple language capabilities. At the core of exceptional voice user interfaces is a real understanding of who your users are and their most desired use cases. Brands and developers should be asking themselves: What are your users’ needs, expectations, and wants? What languages do they speak? What are their privacy concerns? What voice will they connect with the most? Getting the answers to these questions is essential to designing and developing a voice assistant that exceeds user demands and business goals.

To find out more about the essential ingredients that compose a voice AI platform, we interviewed a few voice AI experts we know, asking them for voice AI design tips to help us gain insights into what goes into designing a superior conversational assistant. They talked about everything from user testing, designing for noisy environments, and customizing brand-owned voice assistants to wake words and multilingual capabilities.

Here’s what voice AI experts say about 5 key elements of exceptional conversational AI design:

Know your audience and device
Improve wake word design
Design for your brand with data
Choose the right voice
Make it multilingual

1. Know your audience and device for better voice user interface design

Designing the best user experience possible requires understanding the needs of your users and the functionality goals of the device itself. Any of these elements can alter how you approach designing the conversation flow, the visuals, and even the lexicon of your voice assistant.

For example, if the device is for a car, you’ll want to consider the multimodal aspect for push-to-talk or wake word invocation and decide what information to include for on-screen displays. Making the experience as seamless as possible ensures drivers can find what they’re looking for, whether that’s navigation, entertainment, or answers to queries—without having to take their eyes off the road.

On the other hand, mobile apps and smart TVs will generally have the user looking directly at the screen throughout the entire conversation, but with varying degrees of distance from the device. Smart home technology may not have a visual interface at all.

“The core principles that we use are going to be the same as for visuals—who is the user, and how, when, and where they use it? Then work around the constraints of the interface to come up with a solution for that user. You should think of the user and the device and ask questions. It’s kind of like a dialogue.”
Morris Michael
SoundHound’s Director of Product Design

Want to learn more? Here’s what one of our experts, Morris Michael, SoundHound’s Director of Product Design, has to say about conversational AI design:

Another aspect to consider is whether the device will be used in a noisy environment, such as road noise for a car, street noise for a mobile app, or chatter and music in a home. It’s important to identify the background noise early on to incorporate those sounds into the model so the voice assistant can still function with high accuracy.

“First, you need to understand the use cases in which you intend to use this voice service. Identifying areas of user behavior will help determine what needs to be concentrated on and built to provide a seamless experience for the user.”
Jason Barros
SoundHound’s Director of Partner Program Management

All of these elements are vital to consider early on in the design process as well as throughout every stage. Fine-tuning your voice assistant after it has launched with data and analytics can be a great way to show your users that you’re responsive to feedback and dedicated to creating a positive experience.

Dive deeper into voice user interface design with SoundHound’s Director of Partner Program Management, Jason Barros:

2. Improve your wake word design

Wake words are your users’ first interaction with your brand and voice assistant. Based on those few words, users will create conscious and subconscious impressions about your brand, your voice assistant, and the value of conversational AI.

“You are going to want a wake phrase that is unique and doesn’t rhyme with very many things. The industry standard is to have a minimum of 3 syllables, which ensures you don’t rhyme or overlap with too many other words in your target language. If you do, you will risk opening yourself up to a device responding when you don’t want it to.”
Kayla Regulski

Therefore, wake words are a fundamental piece of the voice assistant that require design expertise and user testing. If the wake word isn’t highly accurate, doesn’t reflect your brand’s values, and fails to create positive impressions with your users by the time it’s launched, it can be extremely difficult to change. Brands serious about connecting with their users and making good first impressions need to ensure their wake phrase will be memorable, easy to pronounce, and efficient.

Curious to dive deeper into wake word design? Hear from the voice AI expert, Kayla Regulski:

3. Design for your brand with data

A voice assistant from a big tech company may have many enticing perks, such as low cost and fast implementation, but there are many long-term costs to consider when voice-enabling your product, app, or service. For starters, your brand becomes subordinate to the big tech’s, with users repeating their brand name every time they wake up the device, not yours. You will also have limited access to data and insights to further improve and iterate your voice assistant. In order to foster brand affinity and have access to data and analytics, you’ll need a custom voice assistant.

“Custom voice assistants are more than an exceptional user experience. They are also an extension of your brand, offering their own personality across many channels. Companies should ask themselves what are the key problems they are trying to solve with a custom voice assistant in order to determine how to approach their voice-first strategy.”
Vitaly Yurchenko
SoundHound’s Director of Business Development

With a custom voice assistant, your users will say your brand name through a branded wake word, sometimes multiple times a day, furthering brand recognition as users associate the positive experience they have with your voice assistant to your brand.

A custom voice assistant provides you with access to valuable data and analytics to know what is working and what needs to be enhanced. When you own the voice experience, you can control how data is collected and what it is used for, and communicating your data policies to your customers creates brand trust. If your voice assistant is owned by a big tech company, your user data is not your own and could be sold to advertisers or used for marketing purposes—creating distrust between your users and your brand.

Need more insights on why you should own your brand and data? SoundHound’s Director of Business Development, Vitaly Yurchenko, breaks it down:

Data is also essential for learning about your users and understanding their different demographics, regions, accents, ages, and cultures can all influence the words your voice assistant will use and be able to understand. Even knowing the age of your users can be helpful. For instance, young children will have a very different vocabulary than adults, and people in some cultures might love informal conversation and humor, while others would deem it inappropriate.

To know your users and their needs, wants, and expectations, user testing is critical throughout each step of the design process. Check out what our Senior User Research, Val Schweisberger has to say on the importance of user testing:

4. Choose the right voice for your conversational AI

Humans tend to anthropomorphize or attribute human characteristics or behaviors to an object, and our voices are one thing that makes us most human. When your users hear your conversational AI’s voice, they will make personal connections and attribute emotions to the tone of voice, pitch, accent, and gender of the assistant. The voice, whether friendly and compassionate or professional and direct, should be determined based on your users’ expectations. Getting it right or creating the wrong persona for your voice assistant can make a huge difference in whether or not your users can connect with it and your brand.

“It’s good to start by identifying the identity of your ideal voice. This should ideally include some discussion within your team and some research about the age, gender, and accent of your voice.”
Andrew Richards
SoundHound’s Director of Business Development

User testing is essential for many aspects of your voice assistant, including identifying which languages your users speak and expect your voice assistant to understand. If a majority of your target audience speaks Mandarin, then having a solely English-speaking voice assistant will miss the mark. By making your conversational AI multilingual, you will broaden your target audience to encompass a wider range of users. When choosing a voice AI provider, it’s vital that they have the knowledge and experience to implement the languages your users need.

Explore the importance of a multilingual voice assistant with SoundHound’s Senior Natural Language Engineer, Monika Depeyrot:

5. Make your voice assistant multilingual

Your users will feel most comfortable when they are able to speak and be understood in their native language. When they feel most at ease is when they will develop the strongest bonds with your voice assistant. Nothing is more frustrating than hearing a voice assistant pronounce words incorrectly or bring up the wrong result because it misunderstood the request.

“There’s a difference between a voice assistant who can speak English or Spanish and a voice assistant who can speak English with a degree of understanding of Spanish. So in California, your English-speaking assistant needs to know non-English words like Vallejo, el Camino real, and jalapeños.”
Monika Depeyrot
SoundHound’s Senior Natural Language Engineer

User testing is essential for many aspects of your voice assistant, including identifying which languages your users speak and expect your voice assistant to understand. If a majority of your target audience speaks Mandarin, then having a solely English-speaking voice assistant will miss the mark. By making your conversational AI multilingual, you will broaden your target audience to encompass a wider range of users. When choosing a voice AI provider, it’s vital that they have the knowledge and experience to implement the languages your users need.

Explore the importance of a multilingual voice assistant with SoundHound’s Senior Natural Language Engineer, Monika Depeyrot:

By knowing your audience and device, improving wake word design, owning your brand and data, choosing the right voice, and making it multilingual, you’ll be able to meet your users’ and brands’ needs while creating meaningful connections and fostering brand affinity. Conversational AI has many required pieces to create a successful user experience. When designing a voice user interface, know the important steps at each stage or risk disappointing your users.
At SoundHound, we have all the tools and expertise needed to create custom voice assistants and a consistent brand voice. Explore SoundHound’s independent voice AI platform at SoundHound.com and register for a free account here. Want to learn more? Talk to us about how we can help bring your voice strategy to life.

Kristen is a content writer with a passion for storytelling and marketing. When she’s not writing, she’s hiking, reading, and spending time with her nieces and nephew.