yes, we are discussing. i personally think we have been on the wrong side of history here and need to figure out a different open source strategy; not everyone at openai shares this view, and it's also not our current highest priority.
That’s a big deal coming from the head of a lab that, while originally founded on the principle of open-sourcing all its research and models, has for the past several years been fully invested in a proprietary AI model, where users can only interact with its software through an application programming interface (API).
It’s important to remember that OpenAI adopted a proprietary model for two reasons. The first was commercial—it's a lot harder to make money from IP you give away, and much harder to maintain any kind of technological advantage over competitors if you let everyone reverse engineer your software. But the second, critically, was about AI Safety. It is a lot harder to prevent people from taking an open source model and using it for harmful purposes. There’s been a lot of discussion in AI policy circles about the critical importance of “securing model weights” as a way to both ensure the U.S. maintains a technological edge over potential adversaries, such as China, and as a way to prevent rogue actors from using AI in dangerous ways.
The business case for open source
The popularity of DeepSeek’s R1 model has altered both aspects of this calculus in some critical ways. First, the marketplace seems to be increasingly voting for open source. Throughout much of 2023, when proprietary models from OpenAI, Anthropic, and Google, were clearly superior in performance, many CTOs and CIOs thought that the only way they could engineer AI systems to achieve business goals was to pay for the more capable proprietary systems.
Since then, however, the free, open weight models have come closer and closer to the performance of the proprietary ones. This was true with Meta’s Llama’s 3 series models, which approach GPT-4o in performance. Now DeepSeek has shown it is also true for the new reasoning models—which are designed to do better at logic problems, math, and coding and which also provide better answers the more time they are given to “think” about a problem. DeepSeek’s R1 comes close to the performance of OpenAI’s o1.
The other thing that has happened is that many companies have discovered that the best way to accomplish business tasks with today’s generative AI systems is to use many models in concert. Engineering this kind of workflow with proprietary models can quickly get expensive. The open source models offer a way for businesses to have more control at potentially lower cost. (Although with open source models, a company still must pay to have the model hosted somewhere.)
Given the narrowing gap in performance between proprietary and open source models and the preference of many companies for open source, it will be difficult for OpenAI, Anthropic, and Google, to hold on to market share from selling foundation models. They may be forced to move “up the stack,” offering their models for free, but then selling ready-made applications and tooling built on top of the models. (This is the traditional business model for open source software providers.) Kevin Weill, OpenAI’s chief product officer, said in the same Reddit AMA in which Altman made his “wrong side of history” remark that OpenAI might open source some of its older models, although whether that would do anything to address the business challenge open source presents OpenAI remains unclear.
The security case against open source
When it comes to safety and security, the growing popularity of powerful open source models also present a dilemma. Open weight models remain fundamentally less secure than proprietary ones. Researchers from the University of Pennsylvania and Cisco published a study last week in which they prodded DeepSeek's chatbot using 50 common jailbreaking techniques—using prompts that try to trick a model into overcoming its guardrails and outputting potentially harmful content. It failed every single one of them, according to the study. That means all those companies rushing to put DeepSeek into their systems, may be unwittingly incorporating some big security vulnerabilities. It also means that bad actors may be able to easily use DeepSeek’s models to help them create malware and cyberattacks, run phishing scams, or even perhaps, plot a terrorist attack. That’s why more money and more effort should be put into figuring out ways to defend open source models. (Anthropic may have hit upon one idea. See more in the Eye on AI Research section below.)
DeepSeek also shows that it is difficult to prevent competing nations from acquiring cutting edge, or near cutting edge AI capabilities. The question then becomes what to do about it. Some critics of U.S. export controls have argued that DeepSeek’s ability to create models as capable as its V3 LLM and R1, despite having access to fewer cutting edge computer chips due to U.S. export controls, show the export controls don’t work. On the contrary, others, such as former OpenAI policy researcher Miles Brundage, argue that export controls are more necessary than ever. His argument is that while export controls may not be totally effective, people would still rather have more powerful AI chips than not, both for developing leading models, and, critically, for running them. So the restrictions should still make it more difficult for China than if they had access to all the Nvidia’s GPUs they might want.
Squaring these security concerns with the business momentum behind open source models will be difficult. Altman may feel OpenAI’s been “on the wrong side of history”—but then again, is he willing to risk recklessly helping to bring about the end of history just to burnish his company’s popularity with developers?
With that, here’s more AI news.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
Hello and welcome to Eye on AI. In this edition: Sam Altman wants OpenAI to rethink its position on open source…OpenAI unveils a clutch of new models and new deals…Anthropic develops a new way to protect AI models from jailbreaking…AI’s big impact on big science.
After last week’s market turmoil around the rise of China-based AI company DeepSeek, OpenAI introduced several new models and features. It’s unclear if any of these releases were accelerated in order to counter the perception that DeepSeek was out innovating OpenAI and other U.S. labs, or whether OpenAI had always planned to make these announcements now. We’ll cover more of this news below, but perhaps the most striking statement OpenAI CEO Sam Altman made last week came in an “ask me anything” session on Reddit, where, in response to a question about whether OpenAI was considering releasing open source models, Altman responded:
yes, we are discussing. i personally think we have been on the wrong side of history here and need to figure out a different open source strategy; not everyone at openai shares this view, and it's also not our current highest priority.
That’s a big deal coming from the head of a lab that, while originally founded on the principle of open-sourcing all its research and models, has for the past several years been fully invested in a proprietary AI model, where users can only interact with its software through an application programming interface (API).
It’s important to remember that OpenAI adopted a proprietary model for two reasons. The first was commercial—it's a lot harder to make money from IP you give away, and much harder to maintain any kind of technological advantage over competitors if you let everyone reverse engineer your software. But the second, critically, was about AI Safety. It is a lot harder to prevent people from taking an open source model and using it for harmful purposes. There’s been a lot of discussion in AI policy circles about the critical importance of “securing model weights” as a way to both ensure the U.S. maintains a technological edge over potential adversaries, such as China, and as a way to prevent rogue actors from using AI in dangerous ways.
The business case for open source
The popularity of DeepSeek’s R1 model has altered both aspects of this calculus in some critical ways. First, the marketplace seems to be increasingly voting for open source. Throughout much of 2023, when proprietary models from OpenAI, Anthropic, and Google, were clearly superior in performance, many CTOs and CIOs thought that the only way they could engineer AI systems to achieve business goals was to pay for the more capable proprietary systems.
Since then, however, the free, open weight models have come closer and closer to the performance of the proprietary ones. This was true with Meta’s Llama’s 3 series models, which approach GPT-4o in performance. Now DeepSeek has shown it is also true for the new reasoning models—which are designed to do better at logic problems, math, and coding and which also provide better answers the more time they are given to “think” about a problem. DeepSeek’s R1 comes close to the performance of OpenAI’s o1.
The other thing that has happened is that many companies have discovered that the best way to accomplish business tasks with today’s generative AI systems is to use many models in concert. Engineering this kind of workflow with proprietary models can quickly get expensive. The open source models offer a way for businesses to have more control at potentially lower cost. (Although with open source models, a company still must pay to have the model hosted somewhere.)
Given the narrowing gap in performance between proprietary and open source models and the preference of many companies for open source, it will be difficult for OpenAI, Anthropic, and Google, to hold on to market share from selling foundation models. They may be forced to move “up the stack,” offering their models for free, but then selling ready-made applications and tooling built on top of the models. (This is the traditional business model for open source software providers.) Kevin Weill, OpenAI’s chief product officer, said in the same Reddit AMA in which Altman made his “wrong side of history” remark that OpenAI might open source some of its older models, although whether that would do anything to address the business challenge open source presents OpenAI remains unclear.
The security case against open source
When it comes to safety and security, the growing popularity of powerful open source models also present a dilemma. Open weight models remain fundamentally less secure than proprietary ones. Researchers from the University of Pennsylvania and Cisco published a study last week in which they prodded DeepSeek's chatbot using 50 common jailbreaking techniques—using prompts that try to trick a model into overcoming its guardrails and outputting potentially harmful content. It failed every single one of them, according to the study. That means all those companies rushing to put DeepSeek into their systems, may be unwittingly incorporating some big security vulnerabilities. It also means that bad actors may be able to easily use DeepSeek’s models to help them create malware and cyberattacks, run phishing scams, or even perhaps, plot a terrorist attack. That’s why more money and more effort should be put into figuring out ways to defend open source models. (Anthropic may have hit upon one idea. See more in the Eye on AI Research section below.)
DeepSeek also shows that it is difficult to prevent competing nations from acquiring cutting edge, or near cutting edge AI capabilities. The question then becomes what to do about it. Some critics of U.S. export controls have argued that DeepSeek’s ability to create models as capable as its V3 LLM and R1, despite having access to fewer cutting edge computer chips due to U.S. export controls, show the export controls don’t work. On the contrary, others, such as former OpenAI policy researcher Miles Brundage, argue that export controls are more necessary than ever. His argument is that while export controls may not be totally effective, people would still rather have more powerful AI chips than not, both for developing leading models, and, critically, for running them. So the restrictions should still make it more difficult for China than if they had access to all the Nvidia’s GPUs they might want.
Squaring these security concerns with the business momentum behind open source models will be difficult. Altman may feel OpenAI’s been “on the wrong side of history”—but then again, is he willing to risk recklessly helping to bring about the end of history just to burnish his company’s popularity with developers?
With that, here’s more AI news.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
