Jensen Huang of Nvidia gave a sneak peek at what the trillion-dollar GPU company is planning to do with future iterations of Deep Learning Super Sampling (DLSS). During a Q&A session at Computex 2024 (reported by More Than Moore), Huang answered a DLSS-related topic, saying that in the future, we will see generated textures and objects that will be created purely through AI. Huang also stated that AI NPCs will also be generated purely through DLSS.
Generating in-game assets with DLSS will help boost gaming performance on RTX GPUs. Work transferred to the tensor cores will lead to less demand on the shader (CUDA) cores, freeing up resources and boosting frame rates. Huang explains that he sees DLSS generating texture and objects by itself and improving object quality, similar to how DLSS upscale frames today.
We could be somewhat close to this next iteration of DLSS technology. Nvidia is already working on a new texture compression technology that takes into account trained AI neural networks to significantly boost texture quality while retaining similar video memory (VRAM) demands of modern-day games. Traditional texture compression methodologies are limited to a compression ratio of 8x, but Nvidia's new neural network-based compression tech can compress textures up to a ratio of 16x.
This tech should apply to Huang's discussion of enhanced object image fidelity through DLSS. In-game objects are just textures wrapped in a 3D space, so this texture compression tech will inevitably boost texture quality.
The more intriguing aspect of Huang's future iteration of DLSS is in-game asset generation. This enhancement of Nvidia's DLSS 3 frame generation tech generates frames in between authentic frames to boost performance. Asset generation is a step beyond DLSS 3 frame generation, with in-game assets generated entirely from scratch through DLSS. (DLSS will need to be told where assets need to be placed in the game world and what assets need to be rendered, but they will be generated (created) entirely from scratch.)
Huang also discussed the future of DLSS surrounding NPCs. Not only does Huang expect DLSS to generate in-game assets, but he also envisions DLSS generating NPCs. He gave an example of six people existing in a video game; two of the six are real characters, while the other four are generated entirely by AI.
It is a callback to Nvidia ACE, which was demoed in 2023. ACE is an in-game LLM designed to bring NPCs to life, giving them unique dialogue and responses in conjunction with user interaction from another character in-game. Nvidia believes ACE (or some future form) will play a vital role in PC gaming and be an integral part of DLSS.
It isn't the first time we've heard about DLSS's future capabilities. The tech giant has publicized that it expects the future of PC gaming to be rendered entirely through AI, replacing classic 3D graphics rendering. In the immediate turn, generating specific assets in-game is a step towards this AI-generated future Nvidia envisions.
