Chatbot Challenge: Do You Know What I Mean?


Bill Murray could single-handedly make most chatbots implode. How would a chatbot react to the following exchange from Ghostbusters?

Ray Stantz: We should split up.

Peter Venkman: Good idea. We can do more damage that way.

Or this from Groundhog Day:

Phil: This is the one time where television really fails to capture the true excitement of a large squirrel predicting the weather.

Or this from Caddyshack:

Carl: In the immortal words of Jean Paul Sartre: ‘Au revoir, gopher’.

Sarcasm and other forms of humor are some of the nuances of human communication that present challenges for chatbots. There have been many technological advances that have enabled the enterprise virtual digital assistant (VDA)/chatbot market to reach its current high-water mark. However, there is room for improvement, and in order to penetrate the mass market, enterprise VDAs have to continue to get better at what they do. Most importantly is understanding language context, but others are critical as well, including user context, live agent/chatbot handoffs, discoverability, internal agreement, and fear of failure.

Understanding Language Context

The challenges of context of a conversation and multiple meanings for words and phrases will continue to frustrate enterprise VDAs, limiting their effectiveness in accomplishing some tasks and functions. Artificial intelligence (AI) advances will help mitigate this limitation over time, but it is unlikely to eliminate the issue. According to Tractica’s recent Natural Language Processing report, it is very early days for understanding context:

Currently, machines have a difficult time understanding humans. The reason for this is context. It is difficult to understand the difference between what someone says and what they mean, and computers are very literal. Understanding language requires the highest form of intelligence. It requires the understanding of location, tone, implied reference, history, and more. Sarcasm and emotion are hard for computers to interpret. Conversational and relational history is hard for computers to interpret. Humans who know each other may reference points of communication on a continuum of their relationship that can span a lifetime.

Researchers at Tsinghua University in Beijing created an “emotion classifying” algorithm for detecting emotion, based on social media posts that had been classified by humans as sad, happy, and other descriptors. According to an article that appeared in The Guardian on May 7, 2017, “The emotion classifier was then used to tag millions of social media interactions according to emotional content. This huge dataset served as a training ground for the chatbot to learn both how to answer questions and how to express emotion. The resulting program could be switched into five possible modes – happy, sad, angry, disgusted, liking – depending on the user’s preference.” The goal is to get the chatbot to learn the appropriate emotion to use for responses. The challenge is our emotional complexity. Humans are motivated differently; one person may need tough love, while another needs encouragement.

Sarcasm and irony are sentiments that are particularly problematic for computers to understand. Sarcasm is the art of saying or writing the opposite of what you mean to insult someone, show irritation, or to be funny. It is frequently seen in user-generated content, particularly social media. Humans can usually detect sarcasm by the way something is said (emphasis, tone, volume, etc.) or in the case of written text, by visual cues and/or conversational history. Researcher Lotem Peled introduced an algorithm designed to help interpret sarcasm for machines in a research paper published in April 2017. The algorithm used a dataset of 3,000 sarcastic tweets (as interpreted by human judges) to train itself to target what they called “sentiment words.” The paper admits this is only the start of an attempt to automatically recognize sarcasm.

User Context

VDAs are challenged to develop user context, such as personal history or location. There are technological issues, such as access and analysis of scattered or unstructured data, and there are privacy concerns around protected personal history and data.

Automated or Live Agent?

Companies are struggling and will continue to struggle to judiciously deploy automation at the right time and for the right situation. Early best practices for enterprise VDAs are one of two flavors. The first scenario is the hybrid automated/live approach in which automated and live agents can seamlessly hand off tasks to each other. Examples would include the ability for an enterprise VDA to start an interaction, but should the interaction veer outside of its expertise, it can be automatically be escalated to reach a live agent. The reverse scenario includes live agents who may start to work through a complex issue, but as the resolution moves in a certain direction, such as gathering personal details for onboarding, then an automated enterprise VDA can take over.

When companies get these roles and handoffs wrong, it can cause great damage and will impact further market adoption of enterprise VDAs.

McKinsey Matrix for Automated and Live Customer Interaction Roles

Discoverability-Consumer Awareness

Tractica estimates that fewer than 1,000 active enterprise VDAs were deployed as of November 2017. In terms of market penetration of consumer-facing businesses, this is a small percentage. Most companies that have deployed enterprise VDAs are larger companies with extensive investment in customer support and call centers, primarily in North America and Europe. As chatbots become more prevalent, companies will have to work hard during an introductory time span to create awareness of chatbots and in what channels they will be available.

Internal Agreement and Integration

Enterprise VDA vendors are finding that most companies understand the value of integrating VDAs into multiple key backend systems. But in reality, integration with multiple systems is a challenge and many result in limited access to data that is needed for complex tasks. Companies interviewed by Tractica cite the typical battles that happen when different departments, such as information technology (IT), marketing, finance, etc., must agree on backend integrations.

Fear of Failure

In the course of interviewing companies for our upcoming VDA report, Tractica found repeatedly that many companies that have developed enterprise VDAs have developed VDAs for internal use first, out of a fear of damaging their business with a customer-facing enterprise VDA. The risk of getting a customer-facing interface wrong is simply too great for many enterprises, slowing the market adoption of enterprise VDAs.

Despite these issues, chatbots will continue to gain contextual intelligence and market acceptance because the market drivers are far too compelling and outweigh the barriers. Savvy companies will continue to narrowly focus the roles of their chatbots and not expect them to be all things to all people.

Comments are closed.