Rivalling models from the two Chinese tech giants have achieved record-high scores on the Chinese Language Understanding Evaluation benchmark. While machines can do better than humans in some language tasks, researchers say AI still has a long way to go before it can truly understand languages. — SCMP
Artificial intelligence (AI) models from Chinese tech giants Tencent Holdings and Alibaba Group Holding understand the Chinese language better than humans, according to a benchmark test measuring natural language processing (NLP).
The two rival models have achieved record-high scores on the Chinese Language Understanding Evaluation (CLUE) benchmark, which is a set of tasks designed to evaluate how well a machine can understand and respond to Chinese text similar to how humans do.
This marks the first time that AI models have scored better than humans on CLUE since the benchmark was established by dozens of researchers three years ago.
Tencent’s “Hunyuan AI model” came first with a score of 86.918, followed by Alibaba’s AliceMind with a score of 86.685, according to Monday’s rankings on CLUE’s website. Both ranked higher than humans, which have been given a score of 86.678.
AI models from Chinese smartphone maker Oppo and food delivery giant Meituan ranked fourth and fifth, respectively.
“Although the rankings change a lot, the top scorers had never surpassed humans until now,” said Alibaba Cloud – the cloud computing arm of the ecommerce giant – in an article published on Friday. It added that the latest results mean that “the Chinese language understanding of the AI model has reached a new level”.
Alibaba owns the South China Morning Post.
China’s Big Tech firms have been working to improve their NLP technology, which is used to support voice-enabled virtual assistants, such as Alibaba’s AliGenie and Tencent’s Xiaowei, as well as other functions like machine translation and spam detection.
Earlier this year, Chinese internet search giant Baidu said that its bot Du Xiaoxiao wrote an article that scored higher than most students on China’s notoriously difficult national college entrance examination, known locally as gaokao, although it mistakenly used an internet slang term.
Still, some researchers say most AI models still have a long way to go before they can truly understand the intricacies of languages.
Last year, scientists at Auburn University in the US state of Alabama and the research unit of American software company Adobe found that many AIs that performed better than humans on certain comprehension tasks were unable to tell when words in a sentence were randomly shuffled. – South China Morning Post
