Nvidia and Microsoft announced on Thursday that they would be adding neural shading support to the Microsoft DirectX preview this April. Neural shading will use cooperative vectors and Nvidia's Tensor cores (matrix operations units) to speed up graphics rendering in games that support the technology. It will better allow for the generic use, via HLSL (high level shading language) of traditional rendering techniques alongside AI enhancements.
While real-time computer graphics and graphics processing units (GPUs) have come a long way, the graphics rendering pipeline itself has evolved slower than hardware. In particular, while Nvidia's GPUs have featured Tensor cores (primarily aimed at AI compute) for over seven years now, they have only been used so far for things like upscaling (Nvidia's DLSS), ray reconstruction (DLSS 3.5) and denoising, and frame generation (at least for DLSS 4).
This is going to change with the so-called neural rendering — a broad term that describes a real-time graphics rendering pipeline enhanced with new methods and capabilities enabled by AI.
A specific subset of neural rendering focused on enhancing the shading process in graphics is called neural shading. Its main purpose is to improve the appearance of materials, lighting, shadows, and textures by integrating AI into the shading stage of the graphics pipeline. The addition of cooperative vectors — which let small neural networks run in different shader stages, like within a pixel shader, without monopolizing the GPU — is a key enabler for neural shading.
Cooperative vectors rely on matrix-vector multiplication, so they need specialized hardware, such as Nvidia's Tensor cores, to operate. To that end, they can work on Intel's XMX hardware as well as Nvidia's tensor cores. Intel also released a statement saying cooperative vector support will be provided on Arc A- and B-series dedicated GPUs as well as the built-in Arc GPUs found in Core Ultra Processors (Series 2) — basically, every GPU from Intel that includes XMX support.
It seems as though cooperative vectors may also work on AMD's RDNA 4 AI accelerators, though RDNA 3 seems more doubtful (as it lacks AI compute throughput compared to the competition). Still, Microsoft is working with AMD, Intel, Nvidia, and Qualcomm to ensure cross-vendor support for cooperative vectors over time.
"Microsoft is adding cooperative vector support to DirectX and HLSL, starting with a preview this April," said Shawn Hargreaves, Direct3D development manager at Microsoft. "This will advance the future of graphics programming by enabling neural rendering across the gaming industry. Unlocking Tensor Cores on Nvidia RTX GPUs will allow developers to fully leverage RTX Neural Shaders for richer, more immersive experiences on Windows."
Nvidia's phrasing makes it sound as though the upcoming DirectX preview with cooperative vectors is as an Nvidia exclusive. However, Intel will co-present a session Cooperative Vectors with Microsoft, so the only unknown right now appears to be AMD GPU support. If driver support is available from AMD, it should work on its GPUs as well.
Ultimately, we'll need to wait to find out not only whether it works, but how well it works — both in terms of image fidelity as well as performance. Differing levels of compute among GPUs will likely affect the end user experience.
