Recently, NVIDIA announced that they have added another offering to their consumer GPU lineup. The entry-level RTX card known as the RTX 2060 comes after the launch of their RTX 2070 and 2080 models. This new lineup of cards has everyone in the gaming space excited, as it brings with it the promise of the holy grail of rendering; dubbed as real-time ray tracing.
But what exactly is ray tracing? This article takes a deeper look into RTX, and how technologies like AI and deep learning and bringing cinematic realism to video games.
What is Ray Tracing?
Ray tracing is a procedure to generate an accurate digital representation of how light rays would affect the visual appearance of an object. This technology has been around since the late 1970s and has since been widely used in Hollywood to render elements in most of their CGI scenes. The procedure aims to bring realistic lighting to simulations by emulating the physical behavior of light.
It is implemented by projecting rays of light from the camera, ‘tracing’ it until it interacts with an object, and calculating where each ray intersects an object in the scene. This then allows it to calculate the way pixel is affected by the ray of light, and continue to trace the ray as it bounces around the scene. This requires the computer to follow some number of rays out from each intersection to model reflected light, thus creating an exponentially increasing number of rays in the scene.
All of these procedures culminate in the calculation of the color of the pixels in the scene, thus providing a realistic look due to light movement. These calculations are compute-intensive, leading to the need for an extremely powerful computer for ray tracing. Each frame of ray-traced content can take hours on a mini-supercomputer class computer, thus making it a technology that Hollywood uses to make CGI in cinemas look realistic. Pixar’s Monsters University utilised ray-tracing extensively, as the technology can reproduce many of the effects created by a camera such as reflections, subsurface scattering, and translucency.
Due to the high amount of performance required to implement ray tracing in a scene, it has long since been considered a goal of technology to implement real-time ray tracing. Video games are an especially difficult space to implement real-time ray tracing, as they output anywhere between 30 to 60 frames in one second. However, NVIDIA has achieved this and has provided a solution that can be implemented in consumer-grade graphics cards with their Turing series of products, which includes the RTX cards.
RTX and real-time ray tracing
To understand how RTX cards are a leap forward in GPU technology, we must first understand how games are rendered today. The work is split mainly between the GPU core and the on-board HBM (high-bandwidth memory). While HBM stores the texture data, such as 3D models, skyboxes, wall textures, and GUI elements, the GPU core calculates how the image should look through a process called rasterization.
Rasterization involves interacting with 3D models created from a mesh of triangles called polygons, which are then mapped into pixels for display on a 2D display. This, along with the shading and processing of pixels, is conducted by the GPU core. As this technique has been the industry standard for video games since the early 2000s, there have been multiple advancements made in order to optimise the experience.
As of now, optimisation in video games is almost a perfect art, with industry standard practices becoming prevalent in almost every prominent title. RTX, however, does not aim to turn all of this on its head. After all, it was NVIDIA who established a majority of these norms in the first place. Instead, the RTX series cards bring optimisations to both existing rasterization-based cards, along with an extra module for real-time ray tracing known as RTCores. These cores act as discrete processing for ray tracing in real time and calculate the various compute-power intensive processes required for ray tracing.
In order to be accessible to consumers, these cards come with a host of optimisations that allow them to achieve ray tracing. Through software, the cards detect how many reflections it needs to follow for a specific ray, depending on the effect on the scene as a whole. There is also intensive post-processing which utilises AI, and a supersampling technique powered by deep learning to offset the performance loss caused by enabling RTX.
The highest end card offered in the RTX lineup currently is the RTX 2080Ti, which can render 10 GigaRays/s. NVIDIA predicts that upcoming solutions will also employ similar hybrid renderers that combine both rasterization and ray tracing, requiring tight integration with existing APIs. This is achieved by another bunch of accompanying software and APIs to enable it to reach maximum adoption among developers and users alike.
NVIDIA has already worked with Microsoft closely in order to bring RTX to the widely used DirectX API for creating games. They have also collaborated with Khronos Group to bring this technology to the Vulkan platform, which is said to be coming soon. Reportedly, a “cross-vendor ray tracing capability” is possible, as NVIDIA is contributing the design to the Khronos Group.
The State of RTX
RTX is supported in 10+ games currently and does not provide ray-tracing support for other games. However, the technology is set for future adoption due to the open API standards and tight integration that Nvidia is bringing to the table.
RTX causes a visible performance hit, which is offset by a technology known as DLSS, which stands for deep learning supersampling. In order to create this, Nvidia trains a neural network on pre-release game engine images at lower and higher resolutions. The AI provides the weights information for Tensor Cores in consumer GPUs through driver updates.
When DLSS is turned on, the game is rendered at a lower resolution, with Tensor cores working to upscale to a higher resolution using Deep Learning. This results in a higher frame rate with a slightly worse image at high resolution. This can be used in conjunction with ray tracing to provide better framerates. NVIDIA claims that users can achieve performance similar to ray tracing off with a combination of DLSS and ray tracing on.
The introduction of this algorithm is required to have playable performance in games, as there is a need to reduce noise in images delivered by ray tracing. This is due to the fact that an extremely high number of rays cannot be cast, leading to a lot of noise in the image. The algorithm functions on the Tensor cores in the GPUs, providing a fast and quality output without sacrificing performance. NVIDIA stated, “These are early days for AI being applied to graphics and the above results are promising”, leading many to believe that AI is the next step in improving graphics performance.
Demos
NVIDIA released a variety of demos that showcase the capabilities of real-time ray tracing in games. Here are some examples to illustrate what difference RTX actually makes in real-world applications.
