Neural Rendering: A New Paradigm in 3D Graphics Programming

In the constantly advancing landscape of 3D graphics, neural rendering technology represents a significant evolution. Neural rendering broadly defines the suite of techniques that leverage AI/ML to dramatically transform traditional graphics pipelines. These new methods promise to push the boundaries of what’s possible in real-time graphics. DirectX is committed to cross-platform enablement of neural rendering techniques, and cooperative vectors are at the core of this initiative.

We are excited to share our plans to add cooperative vector support to DirectX, which will light up cross-platform enablement of the next generation of neural rendering techniques.

What are Cooperative Vectors, and why do they matter?

Cooperative vector support will accelerate AI workloads for real-time rendering, which directly improves the performance of neural rendering techniques. It will do so by enabling multiplication of matrices with arbitrarily sized vectors, which optimize the matrix-vector operations that are required in large quantities for AI training, fine-tuning, and inferencing. Cooperative vectors also enable AI tasks to run in different shader stages, which means a small neural network can run in a pixel shader without consuming the entire GPU. Cooperative vectors will enable developers to seamlessly integrate neural graphics techniques into DirectX applications and light up access to AI-accelerator hardware across multiple platforms. Our aim is to provide game developers with the cutting-edge tools they need to create the next generation of immersive experiences.

What’s Next For Neural Rendering?

The HLSL team is working with AMD, Intel, NVIDIA, and Qualcomm on bringing cross-vendor support for cooperative vectors to the DirectX ecosystem. Stay tuned for more updates about cooperative vectors and its upcoming Preview release!

Cooperative vectors will unlock the power of Tensor Cores with neural shading in NVIDIA’s new RTX 50-series hardware. Neural shaders can be used to visualize game assets with AI, better organize geometry for improved path tracing performance and tools to create game characters with photo-realistic visuals. Learn more about NVIDIA’s plans for neural shaders and DirectX here.