Google's AI scientists have unveiled Meena. Tech watchers are calling it a chatbot breakthrough. Points for responses well matched to human intent. Points for relevant word choices. Points for (gasp) sounding sensible.

Voicebot.AI nailed the goals of this Meena team's effort. You could not get closer to what frustrated users of chatbots wished would come down from the sky. "The scientists behind Meena built the chatbot to be responsive to people's messages, to stay on topic, and to behave as much like another human being as possible."

The article also nailed why Meena stands apart from well known voice assistants: "Meena can theoretically talk about anything, not just the topics already programmed into it."

Kudos from other tech watchers were about conversation delivered at a level that felt more like talking to another person than any existing chatbot.

The Quint: Google has come out with its own chatbot called Meena, and "Early signs suggest the search giant is onto something that could change the dynamics of chatbots in the industry."

Neowin: "Google's AI-based chatbot potentially surpasses all other chatbots available today."

ZDNet said the program "has few of the absurd, nonsensical statements that have tended to characterize chatbots to date. It stays pretty much on topic and is responsive to details in conversation, as rated by human reviewers."

Douglas Heaven in MIT Technology Review: "Open-ended conversation that covers a wide range of topics is hard, and most chatbots can't keep up. At some point most say things that make no sense or reveal a lack of basic knowledge about the world. A chatbot that avoids such mistakes will go a long way toward making AIs feel more human, and make characters in video games more lifelike."

A Google AI Blog entry by the Google Research, Brain Team, explored this concept of open-domain:

"To better handle a wide variety of conversational topics, open-domain dialog research explores a complementary approach attempting to develop a chatbot that is not specialized but can still chat about virtually anything a user wants."

This, they said, can lead to fascination applications: "such as further humanizing computer interactions, improving foreign language practice, and making relatable interactive movie and videogame characters."

The people behind Meena have given themselves bragging rights about which they have not held back. Their paper is titled "Towards a Human-like Open-Domain Chatbot," and it is up on the arXiv server.

"Our contributions are...proposing a simple human evaluation metric for multi-turn opendomain chatbots that captures basic, but important attributes of human conversation...demonstrating that an end-to-end neural model with sufficiently low perplexity can surpass the sensibleness and specificity of existing chatbots that rely on complex, handcrafted frameworks developed over many years."

Google's Meena chatbot scores low on 'perplexity' and, in this instance, low is a positive; Meena has less of a hard time finding the right word.

The team stated that "We present Meena, a multi-turn open-domain chatbot."

S Aadeetya in The Quint explained what Google meant by calling Meena an open-domain platform. Users can type and ask queries from any platform and have a conversation without restricting its expertise.

Those reviewing the paper on arXiv often highlighted Meena's Sensibleness and Specificity Average (SSA).

What is the SSA? Meena was tested with a "human evaluation" metric, this SSA. It captures key elements of a human-like "multi-turn conversation," as described in the paper.

They said that "a human-level SSA of 86% is potentially within reach if we can better optimize perplexity. Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79% SSA, 23% higher in absolute SSA than the existing chatbots we evaluated."

Ofer Ronen, Chatbase general manager, Google's Area 120, appeared in a ZDNet interview to explain what chatbots have achieved and how they have not always fulfilled user needs. Humans like to talk, to use speech, and that alone presents a problem in the man-to-machine age. There are many ways to ask for the same thing, said Ronen.

Tiernan Ray in ZDNet nonetheless made it clear that Meena achieved much but the conversation experience can be expected to be smooth but not scintillating.

"Humans were employed as crowd workers to rate each such conversation for its 'sensibility' and its 'specificity,' and such examples do, indeed, make big progress from prior chatbots. ...Sadly, the humans were not asked by Adiwardana and colleagues to rate conversations for 'interestingness,' because this and other exchanges in the sample are incredibly dull."

But one should not leave it at that, because it appears that the team has realized its steps ahead, as they review their work and what goals are yet to be achieved. They wrote in their paper:

"Furthermore, it may be necessary to expand the set of basic human-like conversation attributes being measured beyond sensibleness and specificity. Some directions could include humor, empathy, deep reasoning, question answering and knowledge discussion skills. "

This is in the early stage of its development, and Meena will undergo further evaluation before you can actually talk with it, said reports.

How was Meena trained? MIT Technology Review referred to a real "data slurp" as Meena was trained on 341 gigabytes of public social-media chatter. Neowin elaborated on their "training on 341GB of social-media conversations and sporting a 2.6 billion parameter end-to-end trained neural conversational model."

---

---

© 2020 Science X Network