Chatbots have enormous application potential in customer service and in sales. And Facebook has just made a big profit in the fight for a better chatbot.
Researchers at the social media giant have created a chatbot that can have extensive, open conversations that are more human than any existing software, the company said on Wednesday.
"This is the first time a chatbot has learned to combine important conversation skills – including the ability to accept a person, discuss almost any topic, and show empathy," Facebook said in a blog post.
In fact, judges hired through Amazon's Mechanical Turk freelance service said they loved talking to Facebook's chatbot almost as much as they did to a real person. The evaluators indicated that in 49% of cases they preferred short conversations with the chatbot when they were in the pits against a similar human-to-human dialogue. "This shows that we're pretty close to human-level performance," said Stephen Roller, one of the Facebook engineers who worked on the project.
Most of the commercially available chatbots, such as B. well-known digital assistants such as Amazon Alexa or Apple Siri are designed so that they are familiar with dialogues on a number of specific tasks: inform you of the weather forecast or give you instructions to the nearest post office. These are the “skills” that Amazon, for example, is constantly expanding on Alexa.
The type of chatbot that the Facebook researchers created is different. It is called "Open Domain Chatbot" and is supposed to be able to have a conversation on any topic. "It can literally talk to you about anything – whether you ate breakfast this morning or which grain is healthiest to feed your kids to your favorite sports team," said Roller.
The chatbot, which Facebook Blender calls because it can “mix” different skills required for successful conversations, has also been tested on dialogs created by a previous chatbot called Meena, which was used by Facebook's cross-silicon earlier this year Valley was created rival Google.
Blender blew Meena out of the water. 67% of the reviewers rate Blender as more human and 75% say they would rather have a long conversation with Blender than with Meena.
Although powerful chatbots have obvious commercial applications, Facebook had no immediate plans to turn Blender into a product. However, the company already has chatbot interfaces that can be used by third parties in its messenger messaging application, and the WhatsApp messaging platform also allows businesses to use chatbots. "This is pure research at this point," said Emily Dinan, another research engineer who worked on the project. "We don't currently want to produce it."
Courtesy of Facebook
Facebook researchers warned that Blender still "has many weaknesses compared to humans," including cases where it contradicts previous statements, repeats itself, or even creates inaccurate factual information. These problems are likely to become more apparent the longer a conversation takes. The benchmark studies were carried out using dialogues consisting of 14 "turns" or reciprocations between interlocutors.
The chatbot can only remember information that spans multiple rounds of conversation, and is therefore more likely to repeat itself in longer conversations, Dinan said.
The Blender chatbot has also only been trained to be used in English, and Dinan admitted that other languages may face greater challenges in creating a chatbot that can handle the proper use of formal and informal tenses and honors with the same fluency as humans put.
The recent breakthroughs in natural language processing – the type of A.I. that can analyze and manipulate language – was the result of using algorithms that collect huge amounts of data about the relationships between words and train them on very large amounts of text samples.
The new Facebook chatbot is no exception. It uses an algorithm that can look at 9 billion variables. It is so large that the neural network, a type of artificial intelligence software loosely based on how the human brain works, cannot fit on a single computer device. Instead, the workload must be distributed across multiple machines that process information in parallel. (The company also created a smaller version that takes 2.7 billion variables into account.)
The chatbot, which uses a software design that was first developed by Google in 2017, has also been trained on a variety of examples. In this case, Facebook used 1.5 billion examples of dialogues from Reddit.com discussion groups to give its algorithm a first basis for how the conversation language works.
It is crucial, however, that Facebook's real innovation was to optimize the software on four smaller data sets. One, referred to as the Wizard of Wikipedia, trains the chatbot to convey factual information from the online encyclopedia, display expert knowledge, and answer certain factual questions. Another, called PersonaChat, teaches the algorithm how to emulate a particular character and how to include information about that character's personal background in a dialog. A third module called Empathetic Dialogues helps the algorithm, as the name suggests, to learn how to recognize emotions and react sensitively.
After the skills of the chatbot have been trained individually for each of these models, they are perfected using a new data set called "Blended Skills Training", in which all three skills of the previous training are integrated. In this way, the chatbot can recognize changes in the tone of the human conversation partner and adapt to them, e.g. B. the change from joking to serious. It also learns when it is best to mention Little Rock is the capital of Arkansas and when it is better to talk about how much German Shepherd dogs like it, or how sorry it is that your goldfish died is.
So far, tech companies have sometimes had trouble making open domain chatbots publicly available. Microsoft researchers released an open domain chatbot called Tay in 2016. The researchers thought the chatbot would perfect its conversation skills by interacting with users. Instead, users soon managed to teach the chatbot to make racist comments.
Dinan said that Blender researchers were well aware that they may have learned racist or sexist conversations, especially through pre-training on Reddit dialogues. However, she said the researchers found that fine-tuning the blended skill talk data reduced the risk of the chatbot making offensive comments. She said Facebook was also looking for systems that would automatically detect and filter offensive language that could be applied to Blender's output in the future.
Other important Fortune technical reporting:
– Who is AT & T's new CEO, John Stankey?
– Tips for working from home from the management team that brought you zoom
– Is A.I. better at diagnosing diseases than doctors? Don't believe all the hype
—Facebook debuts zoom-like video chat feature called Messenger Rooms
– Next, listen to leadership, a fortune podcast that examines the evolving role of the CEO
—WATCH: The ups and downs of zoom since the coronavirus crisis
Find out about Data Sheet, Fortune's daily overview of technology business.