With the rapid emergence of generative AI, it’s becoming more apparent that the technology requires vast resources to scale greater heights. For instance, OpenAI reportedly spends up to $700,000 daily to run ChatGPT. This is on top of the exorbitant amount of water required for cooling per query, with GPT-4 consuming up to four times more water for cooling than previously thought—three water bottles to generate a mere 100 words.
The ridiculous wads of cash required to foster sophisticated AI advances might have heavily contributed to OpenAI’s bankruptcy reports from last year, with projections of making a $5 billion loss within one year. However, a new Chinese AI startup, DeepSeek, is redefining these odds with a new flagship model dubbed DeepSeek-R1, which can compete with OpenAI’s o1 reasoning model on an even playing field, potentially giving top AI labs, including OpenAI, Anthropic, and Google a run for their money.
As you may know, multiple reports have surfaced online, alluding that AI progression has been stunted by scaling laws, making it difficult for top AI labs to develop advanced models due to a lack of high-quality content for training. Former Google CEO Eric Schmidt and OpenAI CEO Sam Altman dismissed the claims, indicating no sign scaling laws have begun as AI models are expected to scale unprecedented heights. “There’s no wall,” added Altman.
What makes DeepSeek-R1 special compared to competitor AI models?
As you might have guessed, R1’s earlier comparison highlights that it shares similar capabilities with OpenAI o1 across math, coding, and reasoning. According to the research paper published by the AI firm last week, the model’s astonishing reasoning capabilities are based on a sophisticated technique dubbed pure reinforcement learning. NVIDIA’s senior research manager, Jim Fan, claims the technique is similar to Google DeepMind’s approach with AlphaZero, which has showcased its prowess across a wide range of games, including go and chess "without imitating human grandmaster moves first"(via Business Insider).
What exactly does R1 do? DeepSeek says it achieves "performance comparable to OpenAI o1 across math, code, and reasoning tasks."
Interestingly, R1 scored 79.8% on the AIME 2024 mathematics benchmark, beating OpenAI’s o1 reasoning model. It further demonstrated expert level in coding tests with a 2,029 Elo rating on Codeforces and outperformed 96.3% of human competitors.
Reinforcement learning fosters powerful and interesting reasoning behaviors, allowing the model to develop advanced capabilities like self-verification and reflection without prior training or programming.
We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive - truly open, frontier research that empowers all. It makes no sense. The most entertaining outcome is the most likely.DeepSeek-R1 not only open-sources a barrage of models but… pic.twitter.com/M7eZnEmCOYJanuary 20, 2025
We are living in a timeline where a non-US company is keeping the original mission of OpenAI alive - truly open, frontier research that empowers all. It makes no sense. The most entertaining outcome is the most likely. DeepSeek-R1 not only open-sources a barrage of models but also spills all the training secrets. They are perhaps the first OSS project that shows major, sustained growth of an RL flywheel. Impact can be done by "ASI achieved internally" or mythical names like "Project Strawberry". Impact can also be done by simply dumping the raw algorithms and matplotlib learning curves.
NVIDIA Sr. Research Manager, Jim Fan
For context, DeepSeek-R1 is based on an earlier version of the model called R1-Zero, which prompted an “aha moment” for the AI researchers behind its development. The researchers disclosed that the AI "learns to allocate more thinking time to a problem by reevaluating its initial approach." Wharton Prof. Ethan Mollick says R1’s responses resemble a “human thinking out loud.”
R1’s launch comes as OpenAI and Anthropic are in an arm’s race chasing the coveted AGI benchmark. OpenAI CEO Sam Altman indicated that his team knows how to build AGI and that it would be achieved sooner than anticipated with current hardware. The executive added that the benchmark would whoosh by with surprisingly little societal impact despite a 99.9% probability AI will end humanity.
To that end, DeepSeek's open-source model potentially restores fair competition in the AI landscape. OpenAI shipped its o1 reasoning model to broad availability but buried it under its new $200 ChatGPT Pro plan in December. Sam Altman explained the reasoning behind the model's exorbitant pricing by indicating that it "can think harder for the hardest problems."
According to Deli Chen, DL researcher at DeepSeek:
“Unbelievable results, feels like a dream—our R1 model is now #1 in the world (with style control)! 🌍🏆 Beyond words right now. 🤯 All I know is we keep pushing forward to make open-source AGI a reality for everyone. 🚀”
Fun demos of DeepSeek's new r1 shutting itself down when asked about topics the CCP doesn't like ⬇️But the censorship is obviously being done by a layer on top, not the model itself. Anyone have the open source version running and able to test whether/how much it also censors? https://t.co/gy0pId9Qw0January 20, 2025
Besides open-sourcing its R1 flagship AI model, DeepSeek is breaking the bank to run the model. According to the company’s research paper, the model runs queries at just $0.14 per million tokens, stacking miles behind OpenAI’s steep $7.50. DeepSeek surpassed OpenAI’s exorbitant efforts with its o1 reasoning model with approximately 3-5% of the cost.
Fun demos of DeepSeek's new r1 shutting itself down when asked about topics the CCP doesn't like ⬇️But the censorship is obviously being done by a layer on top, not the model itself. Anyone have the open source version running and able to test whether/how much it also censors? https://t.co/gy0pId9Qw0January 20, 2025
R1 faces its share of challenges, including shutting itself down when asked about topics the CCP doesn’t like, and demonstrating censorship that could lobotomize the model and ruin its user experience. According to former OpenAI board member Helen Toner:
“But the censorship is obviously being done by a layer on top, not the model itself. Anyone have the open source version running and able to test whether/how much it also censors?”
