Shortages of the specialized computer chips needed to run its artificial intelligence software are holding back OpenAI’s business, and the company has no intention of releasing a consumer-facing product beyond ChatGPT. Those are just two of the disclosures OpenAI cofounder and CEO Sam Altman reportedly made to a group of software developers and startup CEOs at a private meeting in London two weeks ago, according to a blog post written by one of the participants. The account of the closed-door meeting, reportedly attended by about 20 people, was later taken down at OpenAI’s request, according to a note appended to the page where it initially appeared, but that hasn't stopped the A.I. community from poring over the influential CEO's (alleged) comments.
An internet archiving site had already saved a copy of the original blog post, and it has since circulated on social media and several coder-oriented discussion boards. Altman said OpenAI’s inability to access enough graphics processing units (GPUs), the specialized computer chips used to run A.I. applications, is delaying OpenAI’s short-term plans and causing problems for developers using OpenAI’s services, according to the blog post penned by Raza Habib, an A.I. expert who is also the cofounder and CEO of Humanloop. Habib's London-based startup has pioneered methods to make the training of large language models, such as those that underpin OpenAI’s ChatGPT, more efficient.
The shortage of GPUs has made it harder for OpenAI to let users push more data through the large language models that underpin its software, such as ChatGPT, and slowed the company's planned rollout of additional features and services. It has also made OpenAI's existing services slower and less reliable, according to the blog post, a fact that is frustrating customers and making them reluctant to build enterprise applications on top of OpenAI's technology. The chip supply crunch has risked OpenAI's first-mover advantage in the generative A.I. boom, as Google—as well as lesser-known rivals—has been able to roll out competing services, and open-source competitors have gained a greater foothold.
All about the 'context window'
Altman laid out several things that OpenAI just can't do yet because it lacks the hardware (i.e., the chips). These include providing a longer “context window” to most customers of its GPT large language models, Habib wrote in his blog post. The context window determines how much data can be used in a single prompt that is fed into the model and how long the model’s response can be. Most users of GPT-4 have a context window that is 8,000 tokens long (a token is a segment of data on which the underlying A.I. model makes a prediction, equivalent to about one and a half words of English). OpenAI announced a 32,000-token window for select users of the model in March, but few users have been granted access to that feature, a fact Altman blamed on the lack of GPUs, Habib wrote.
The majority of the world's A.I. applications are trained and run on GPUs, a kind of computer chip that is designed to crunch data using parallel processing at high speeds. Most of those chips are made by just one company, Nvidia, and can cost thousands to hundreds of thousands of dollars. Market watchers already know that Nvidia's stock has soared due to its association with the boom in generative A.I., and its market valuation recently crossed the $1 trillion threshold.
The OpenAI cofounder and CEO also reportedly assured the developers that OpenAI has no plans to launch any consumer-facing products beyond ChatGPT, according to Habib’s post. Habib had said that many developers at the meeting told Altman they were concerned about using OpenAI’s A.I. models to build upon if OpenAI itself might later roll out competing products. Altman reportedly said ChatGPT would be its only consumer-facing product and that his vision for its future was as a “super smart assistant for work” but that many industry-specific cases involving the underlying GPT large language models OpenAI “wouldn’t touch.”
Altman also reportedly said that comments he had a month ago about “the era of giant models” being over had been wrongly interpreted. The OpenAI chief told developers that he only meant to say that given how large GPT-4, OpenAI’s most powerful large language model, already is, it would not be possible to continue to scale up A.I. systems exponentially. He told the London meeting that OpenAI would continue to create larger models, but they would be only two or three times bigger than GPT-4, not millions of times larger.
In the conversation with developers, Altman also reportedly laid out OpenAI’s near-term road map. Within 2023, Altman said OpenAI’s goals were to make GPT-4 faster and cheaper, provide a longer “context window” to allow people to feed OpenAI’s GPT models more data and receive longer outputs, roll out an easier way to fine-tune GPT-4 for specific customer use cases, and also allow ChatGPT and its underlying large language models to retain a memory of past dialogues, so that one would not have to repeat the same sequence of prompts each time a person wanted to pick up a conversation where they left off or repeat a certain interaction with the model, Habib’s blog post said.
Next year, Altman reportedly said the priority would be to roll out GPT-4’s ability to receive images as inputs and outputs, a feature the company demonstrated when it debuted the model in March, but has not made available to most customers yet.
When it comes to regulation, Altman said to the developers that he did not think existing models posed any outsize risk and that “it would be a big mistake to regulate or ban them,” Habib wrote. Altman reiterated his public stance that OpenAI believed in the importance of open-source A.I. software and confirmed a report from the tech publication The Information that OpenAI is considering open-sourcing one of its models. According to the blog, Altman said the company might open-source its GPT-3 model and only hadn’t done so yet because Altman “was skeptical of how many individuals and companies would have the capability to host and serve” large language models.
Altman reportedly told the closed-door meeting that the company was still trying to figure out how ChatGPT Plus customers wanted to use the plugins that allow the large language model to use other software. Habib said in the blog that this probably meant that the plugins did not yet have product-market fit and would not be rolled out to enterprise customers through OpenAI’s API anytime soon.
Neither Habib nor OpenAI immediately responded to requests for comment from Fortune.
Habib’s blog post inspired heated discussion on social media and developer forums. Many said Altman’s comments showed just how much of a problem the lack of GPUs is for realizing the business potential of large language models. Other said it showed just how vital many of the innovations emanating from the open-source A.I. community—which has developed innovative ways to achieve similar performance to some of the largest proprietary A.I. models using much less computing power and much less data—are to the technology’s future.
Meredith Whittaker, the president of the Signal Foundation and a leading critic of Big Tech, interviewed on the sidelines of a conference in Berlin, said the blog post showed the stranglehold that the world’s largest technology companies hold over the foundations of today’s A.I. software because only these companies can afford the computing resources and data needed to train the largest A.I. models. “What you see is that the primary constraint, even with access to Microsoft’s infrastructure, is GPUs,” she said, referring to OpenAI’s close partnership with Microsoft, which has invested $13 billion into the San Francisco A.I. startup to date. “You need incredibly expensive infrastructure to be able to do this.” She said people should not confuse the fact that an open-source A.I. community exists “with an actually democratic and competitive landscape.”
Fortune reporter David Meyer in Berlin contributed reporting to this story.
