There is panic in the American AI community, a "Sputnik moment", worried Marc Andreessen, investor and close advisor to Donald Trump, in reference to the first satellite launched by the Soviet Union in 1957 that caught the United States off guard. A Chinese chatbot, DeepSeek, has been topping the ranking of the most popular free applications around the world for a few days.
David versus Goliath
The most advanced language model (LLM) proposed by the bot, DeepSeek-R1, would do better than the o1 model, the most advanced from OpenAI, on math and reasoning problems, according to benchmarks carried out by DeepSeek. By default, the chatbot works with DeepSeek-V3, which reportedly cost less than $6 million to develop. A drop in the ocean in the hundreds of billions needed to design high-performance LLMs, as the American giants in the field have constantly reminded us — through the Stargate project, for example.
DeepSeek has this little David defeating Goliath side that could shake the certainties of the US (and European) industry: with its open source models calculated with chips that are less powerful than those of Western champions, the Chinese bot would therefore do better… and even cheaper. According to Epoch AI, DeepSeek’s model would be so efficient that its training would only require a tenth of the computing power of Meta’s Llama 3.1 model.
Who says Chinese bot also implies restrictions on certain “sensitive” requests. For example, it’s impossible to ask DeepSeek questions related to the Tiananmen Square uprising or Xi Jinping.
Since 2021, it’s been very difficult to get the most advanced American silicon in China, due to restrictions imposed by the White House. To be able to continue working, Chinese AI developers are sharing their work with each other and trying new approaches. “DeepSeek focused on optimizing resources through software solutions,” Marina Zhang, an associate professor at the University of Technology Sydney, told Wired.
The Chinese technology expert continues: “DeepSeek adopted open source methods, pooling collective expertise and promoting collaborative innovation. This approach not only overcomes resource constraints, but also accelerates the development of cutting-edge technologies, which sets DeepSeek apart from its more closed competitors.
DeepSeek was founded in 2023 by Liang Wenfeng in Hangzhou. This electronics engineer assembled a battery of 10,000 Nvidia H100 chips that are now banned from import into China. This infrastructure would have been supplemented by less powerful chips that are still available for import, and optimizations to limit costs and reduce the computing power required to train LLMs.
DeepSeek's success is a real electroshock for the Western AI industry. The Chinese example will perhaps push it to work more together, and to seek greater efficiency with existing infrastructures.
Source: Wired
0 Comments