OpenAI expects to finalize its first custom AI processor design in the coming months and send it to TSMC for production, aiming for large-scale manufacturing by 2026, reports Reuters. OpenAI follows its rivals from Google, Meta, and Microsoft, so to remain competitive in terms of costs, it needs its own custom processors earlier rather than later.
The custom silicon OpenAI will create for AI processors is expected to feature a so-called systolic array design, a grid of identical processing elements (PEs that perform matrix or vector computations) that are arranged in rows and columns and connected in such a way that data 'pulses' through the array in a pipeline-like fashion. The processor is said to use HBM memory, though it is unclear whether OpenAI plans to use HBM3E or HBM4. As for process technology, OpenAI reportedly aims at TSMC's proven N3-series (3nm-class) fabrication process.
OpenAI is reportedly working with Broadcom on its custom processor for AI workloads project. Typically, companies that work with Broadcom on custom processors develop key differentiating intellectual property (IP) in-house (or at least define it with Broadcom), and then Broadcom adds the remaining parts such as general-purpose CPU cores, memory, and I/O controllers and physical interfaces, as well as assembling the final design.
On the OpenAI side, the effort is led by Richard Ho, who previously worked on Google's TPUs. Ho's team reportedly doubled to 40 engineers recently, but it is still considerably smaller than those at Amazon Web Services or Google. Expanding the initiative to match the scale of Google or Amazon would require hiring hundreds more engineers. That said, it is reasonable to expect OpenAI's in-house contribution to the design to be relatively small.
Internally, OpenAI sees its custom processor as a way to improve its bargaining power with existing suppliers, mainly Nvidia. However, if successful, the company intends to refine and upgrade its custom silicon over time, expanding functionality with each iteration. The first version is expected to be produced in small quantities, primarily for running inference workloads on AI models rather than training them.
Developing a high-performance AI processor is an expensive undertaking. Industry estimates suggest a single model could cost hundreds of millions of dollars, and supporting infrastructure, including necessary hardware and software, could double that amount. However, when more than one processor is developed per platform, those costs per processor typically drop.
Historically, even companies like AWS, Google, Meta, and Microsoft have struggled to create competitive in-house processors that could beat Nvidia's GPUs in terms of performance. However, they have managed to build much cheaper processors with higher energy efficiency tailored for their workloads, which enables them to more than offset development costs. Also, these custom processors make running AI models cheaper for their cloud customers, which is good for the market.
The demand for AI chips continues to surge as Big Tech companies need vast quantities of processors to train and then run their increasingly sophisticated models. Meta has allocated $60 billion for AI infrastructure this year, while Microsoft plans to invest $80 billion in 2025. OpenAI has not formally announced its 2025 spending plan — which is not surprising as it is not a public company — but, likely, it will also spend tens of billions of dollars on hardware, software, and infrastructure this year.
If OpenAI manages to tape out its first custom processor in the coming months, then it will be able to mass produce it sometime a year after that, in mid-2026. If it is lucky, it will begin deployment in the second half of 2026.
