When AMD launched its Instinct MI300X accelerators for AI and HPC about a year ago, Amazon Web Services (AWS) expressed interest in deploying them in the cloud. However, according to Amazon, as reported by Business Insider, due to a lack of strong demand, the company still has not done so.
"We follow customer demand," Gadi Hutt, Director of Product and Customer Engineering at Annapurna Labs, an Amazon company, told Business Insider. "If customers have strong indications that those are needed, then there's no reason not to deploy."
At least, according to Hutt, there has not been enough interest to justify the deployment of AMD's Instinct MI300X accelerators at AWS. While AMD's Instinct MI300X is cheaper than Nvidia's H100, its software is not as robust as Nvidia's CUDA, which scares off many developers. As AMD's hardware offerings improve (e.g., Instinct MI325X), so too should its software.
To some degree, Hutt may be considered an interested party as Annapurna-developed Trainium rivals those from AMD and Nvidia in AWS's data centers. Still, assuming that he talked on the record, this is AWS's stance.
Speaking of Trainium, with its in-house-designed Trainium and Trainium2, AWS does not have to pay a premium to AMD or Nvidia, which is why it can offer Trn1 and Trn2 instances at very competitive prices compared to those powered by Nvidia's H100 GPUs. This may be another reason for the low interest in non-Nvidia third-party solutions.
Speaking of Nvidia, AWS announced at its re:Invent conference that it was set to strengthen its AI offerings with Nvidia's upcoming Blackwell GPUs for AI and HPC. During the re:Invent conference, AWS introduced its P6 servers equipped with Blackwell GPUs, reflecting the expectation that these machines will be in high demand.
Despite not offering AMD's Instinct MI300X in the cloud, AWS continues to collaborate closely with the company and offers plenty of instances based on AMD's EPYC processors. Given their core count and memory subsystem, these processors offer massive advantages over Intel Xeon rivals for compute—and memory-intensive instances.
