Nvidia’s GPU Technology Conference (GTC) in San Jose this week, offered the usual showcase of the company’s latest innovations in machine learning (AI), autonomous vehicles, scientific and industrial supercomputing, workstation graphics and robotics. Punctuated by Nvidia CEO Jensen Huang’s usual passionate and authentic on-stage delivery, there were a couple of major takeaways from the show that stuck with me, beyond the burly prowess of the silicon valley powerhouse’s revamped Volta-powered AI processors and professional workstation GPUs. In short, the company is about to drop the pedal and hit another gear in the AI race, from a silicon platform standpoint, and here are just a couple of reasons why.
AI Supercomputers Require More Than Just Fast Processors, They Need Superhighways
Nvidia took the wraps off its next generation Volta V100 AI processing engine and on the surface it may have seemed like sort of a ho-hum unveil. The new GPU-based Volta module now supports a full 32GB of HBM2 (High Bandwidth Memory), doubling the multi-chip module’s memory footprint, which is critical to accommodating ever-growing AI datasets, allowing the processor to stay on-chip for computation, versus going out to much higher latency system memory. However, more impressive to me was the introduction of the company’s new DGX-2 Machine Learning supercomputer, which now supports up to double the number of Nvidia Volta GPU modules, now at 16, up from the previous generation’s max of 8. Again, this seems trivial but it’s really not.
Nvidia had to architect a new type of super low-latency switch fabric, it calls NVSwitch, which is based on the company’s previous NVLink technology but allows a completely non-blocking crossbar connection between all processor complexes. Effectively, this allows total processing power to double; simple math with double the processor count of course. However, it also allows for a 4X increase in contiguous memory space, with the Volta modules doubling their memory to 32GB, and the number of modules in the system also doubling with the ability to address and access each other’s memory.
In conversation with some very high-ranking AI execs at Intel — which is also working fiercely competitively in this space, with multiple solutions from FPGAs to its Nervana Neural Network processors — this obstacle of scalability is paramount. The comment was made that system architecture and performance is nearly as important as processor architecture, because if you can’t get on and off chip and in and out of local memory really fast, it’s like having a Ferrari in the city at rush hour – you might look good but you’re going nowhere fast. Side note: Intel is coming on strong here in its own right. More to come on this.
AI Is Naturally A Hive Mind Technology And It’s About Spreading The Love
Nvidia was pretty smart about evangelizing their GPU technologies for AI and machine learning workloads, back in the days when technologies and concepts were significantly more fledgling. Back then, various GPU Compute applications just didn’t have the software tools infrastructure out there, so the company came up with its CUDA programming API and software development toolkits. Then the company distributed these tools like Johnny Appleseed on steroids to every university, researcher, developer it could engage with, from applications in the cloud to everything in between, based on its parallel processing model. As a result, CUDA is now defacto standard in machine learning programming for many in the field. Now it appears, Nvidia is about to make another particularly adept move of seeding the IoT for edge AI-enabled usages models and products.
Again, don’t let the surprisingly quiet announcement of Nvidia’s partnership with ARM deceive you. What you have here is an AI processing juggernaut with a deep, refined toolset that is already very familiar with ARM core processor technology IP, having employed in many of its own silicon designs for variously applications, from self-driving cars to Android TV set top box devices – and that AI powerhouse is now partnering with its own custom core IP called NVDLA, with the absolute industry leader in silicon core licensing models.
“Accelerating AI at the edge is critical in enabling Arm’s vision of connecting a trillion IoT devices,” said Rene Haas, executive vice president, and president of the IP Group, at Arm. “Today we are one step closer to that vision by incorporating NVDLA into the Arm Project Trillium platform, as our entire ecosystem will immediately benefit from the expertise and capabilities our two companies bring in AI and IoT.”
A stroke of brilliance here? It just may be. Deepu Talla, vice president and general manager of Autonomous Machines at NVIDIA notes, “Inferencing will become a core capability of every IoT device in the future… Our partnership with Arm will help drive this wave of adoption by making it easy for hundreds of chip companies to incorporate deep learning technology.”
So here goes Nvidia again with the Johnny Appleseed model. This ought to be interesting to watch, especially for IoT edge applications. However, either way you slice it, from system scalability and performance, to spreading its influence — and by association quite literally its brain-connected synapses with this ARM deal — Nvidia seems to not only have an early lead in the AI processing game, but also a firm grasp of what it takes to keep it.
