Chinese-Built ChatGLM Exceeds GPT-4 Across Several Benchmarks
A recent research paper states, “The latest ChatGLM language model from Tsinghua University and Zhipu AI matches or exceeds the capabilities of GPT-4 across a wide range of benchmarks and tasks.”
The GLM-4 model was pre-trained on 10 trillion tokens of multilingual data and further aligned using techniques like supervised fine-tuning and reinforcement learning from human feedback.
On standard English academic benchmarks spanning knowledge, math, reasoning, and coding, GLM-4 achieves performance comparable to GPT-4 and other state-of-the-art models like Gemini 1.5 Pro and Claude 3 Opus. It scores 83.3% on MMLU (vs 86.4% for GPT-4), 93.3% on GSM8K (vs 92.0%), and 84.7% on the challenging BIG-Bench suite (vs 83.1%).
For instruction following abilities in both English and Chinese, GLM-4 matches the level of GPT-4 Turbo, according to the IFEval benchmark. On Chinese language alignment across domains like math, logic, and professional knowledge, GLM-4 outperforms GPT-4 and other models on the AlignBench evaluation.
The GLM-4 All Tools version can autonomously employ external tools like web browsers, Python interpreters, and text-to-image models to complete complex multi-step tasks. It matches, and in some cases, even surpasses GPT-4 All Tools on capabilities like information gathering and math problem solving.
Tsinghua University has open-sourced several GLM models, with over 10 million downloads in 2023. The team plans to continue improving the model’s capabilities while promoting open access to cutting-edge language AI technologies.
China Introducing Models
China has a history of introducing advanced AI models that manage to surpass their Western counterparts. A few months ago, they released ChemLLM, developed jointly by researchers from Hong Kong Polytechnic University, the Chinese University of Hong Kong, Shanghai Artificial Intelligence Laboratory, Fudan University, Shanghai Jiao Tong University, and Wuhan University. This model is designed to tackle a wide array of chemical tasks through fluent dialogue interaction.
Similarly, by consistently releasing and experimenting with new models, China is advancing rapidly in the field of AI.
