Achieving human parity is the holy grail for any interface system using AI. A term used to denote reaching a human-like processing level, human parity is now beginning to become more and more accessible through algorithms.
One of the biggest places to reach human parity is with chatbots, which present a unique opportunity of interfacing with humans. It has historically been one of the more difficult fields in which to achieve human parity, due to a lot of reasons.
A Tough Ask
Language is one of humankind’s most complex creations. The concept functions on both text and context, which humans can easily understand due to it being fundamentally wired into their mind.
For example, in a conversation about 2 or more individuals, a human can understand whom the other person is talking about by gleaning context from the conversation. However, an algorithm would get confused whom the speaker was referring to.
Added to this, there are multiple meanings to many words in the language, not to mention complex grammatical syntaxes to take into consideration. This becomes a positive maze for an algorithm to navigate through while trying to understand humans.
More than all of this, the very fact that collaqualisms exist makes it extremely difficult for bots to understand certain terms. While these are indicative of the way language has evolved in a certain geographical location, they do not ‘make sense’ outside the cultural context from which they were birthed in the first place.
This leads to a difficult situation for chatbots, as they are not capable of contextualisation of language. They do not possess the cultural or linguistic knowledge required to decode the subtle intricacies of language.
So, how do chatbots work?
Building A Chatbot
Due to the vast reach of language, it is difficult to design a chatbot to suit all applications. Currently, in the scope of technology we have today, a chatbot that exhibits human parity across the entirety of a language is improbable.
Therefore, it is more practical to develop a chatbot for an application in a specific subset of information. For example, a customer support chatbot will be completely different from a banking chatbot.
How a chatbot should be developed will be based on two things, one being the complexity of the work that the bot is required to do, and the other being the complexity of the data that is being entered into it. Based on these two factors, a chatbot can be constructed for specific use cases.
The creators of the chatbot must also understand what role the bot must fulfil. A narrow focus is required, as not all the answers can be programmed manually.
For example, a banking chatbot designed to help users with tasks they might not understand how to perform will provide relevant information in an accessible manner. The information will be restricted to purely banking data, and the chatbot will be built around that.
A user not knowing how to apply for a cheque book, for example, can go to the bot and ask it the question. The bot must have the question programmed into it in the most basic forms, with contextualisation and extrapolation done by the AI part of the bot.
In this example, the question saved will include the keywords how, apply, chequebook. The AI part of the bot will conduct a multitude of tasks to convert the user’s unstructured question into one that can be understood by it. This is known as natural language understanding.
How Chatbots Understand Humans
Chatbots utilise a variety of methods to understand the users’ needs. To process the data in a way that can be used by the chatbot, a process called lemmatization is used. This is a process used to interpret the ‘true meaning’ of a word, in case the user uses the plural form or a synonym.
One of the most commonly used methods to process the data is intent classification. As the name suggests, intent classification is the method used to classify what response needs to be given by the chatbot to a specific query. The response offered by the bot will be according to the classified ‘intent’ of the query.
The bot will also need to be programmed to resolve issues, if it is a customer-facing chatbot. In case of a customer support bot, it will need to be trained to resolve various problems that the customer has.
There are also a multitude of other methods that chatbots utilise that fall under the purview of natural language understanding. However, none of these seems scalable to the point of reaching human parity.
Will Chatbots Ever Achieve Human Parity?
In a country like India, where colloquialisms are rampant and there are multiple dominant languages, it becomes a difficult undertaking to create a chatbot that will exhibit human parity.
It is important to note that the field is continuing to progress, especially with respect to NLP. The level of advancement taking place in the NLP field ensures that a much faster resolution is reached.