Alibaba’s robots are better than humans in reading test
First, they beat us at chess. Then it was Go. Now it’s basic reading comprehension. The robots are coming.
Alibaba says its deep neural network model has outscored humans in a global reading test, paving the way for the underlying technology to reduce the need for human input. The Chinese tech giant’s research unit, Institute of Data Science of Technologies (IDST), said it had developed a deep-learning model that attained a score of 82.44 in Exact Match on the Stanford Question Answering Dataset (SQuAD). Humans had clocked a previous score of 82.304, it said.
A different program built by Microsoft scored higher than Alibaba’s at 82.605 on the same test, but that score was finalized a day later, according to Bloomberg.
The test, known as Stanford Question Answering Dataset or SQuAD for short, asks contestants — human and robot — to provide exact answers to more than 100,000 questions drawn from more than 500 Wikipedia articles. The test is designed to see if artificial intelligence can process large amounts of information before fully comprehending it and offering precise answers.
Some of the Wikipedia articles from which questions were drawn covered a wide range of topics, from Super Bowl 50 (“Where did Super Bowl 50 take place?” Answer: Santa Clara.) to Doctor Who (“What planet is Doctor Who from?” Answer: Gallifrey.).
“These kinds of tests are certainly useful benchmarks for how far along the AI journey we may be,” Microsoft spokesperson Andrew Pickup told CNN. “However, the real benefit of AI is when it is used in harmony with humans.”
Major technology companies in the United States and China have invested billions of dollars in artificial intelligence to gain a foothold in what may be the next technological frontier. The Chinese government has outlined a plan to create a $150 billion AI industry by 2030 in partnership with private companies such as Alibaba and Tencent.
Microsoft in December announced its “AI on Earth” project to help the planet become more environmentally sustainable using the company’s in-house AI infrastructure. Microsoft will invest $50 million over the next five years, according to Microsoft CEO Brad Smith.
“At Microsoft, we believe artificial intelligence is a game changer,” said Smith. “As we enter the world’s Fourth Industrial Revolution, a technology-fueled transformation, we must not only move technology forward, but also use this era’s technology to clean up the past and create a better future.”
The company had said its AI-powered customer service chatbot, Dian Xiaomi, was used to support its online merchants and served an average of 3.5 million users daily across its Taobao and Tmall platforms.
Commenting on the SQuAD score, Alibaba IDST’s chief scientist of natural language processing Si Luo, said: “That means objective questions such as ‘what causes rain’ can now be answered with high accuracy by machines. We believe the technology underneath can be gradually applied to numerous applications such as customer service, museum tutorials, and online responses to medical inquiries from patients, decreasing the need for human input in an unprecedented way.”
Si said Alibaba would be “sharing our model-building methodology” with the community and planned to apply the technology to support its customers in the near future.