At GDC 2023, AMD unveiled FidelityFX Super Resolution v3, or FSR 3.0, a solution for upscaling games to save performance. FSR 3 is a direct competitor to NVIDIA’s deep learning super sampling (DLSS), which uses AI to improve performance in games. 

In the age of raytraced games, boosting frame rates while maintaining quality has become a priority for both AMD and NVIDIA. While the former has been lagging behind the latter due to their late start, it seems that AMD’s FSR 3 can provide a more accessible option for low-end gamers as opposed to NVIDIA’s DLSS, which only works on NVIDIA 20-series GPUs and above. 

Why do gamers need upscaling?

The era of upscaling can be traced back to the launch of DLSS in February 2019. This technology was released along with the first generation RTX 20-series (ray tracing) cards, primarily to make up for the performance loss that comes with enabling ray tracing. 

Real-time ray tracing is a new graphical technology pioneered by NVIDIA which simulates the light rays bouncing off objects in real time. With RTX, games can accurately show lighting, shadows, and reflections in a way that more closely resembles reality.

As one can imagine, calculating the path of individual light rays takes a large amount of computing resources. This is why NVIDIA created RT cores to accelerate ray tracing tasks. However, even with RT cores, enabling ray tracing generally resulted in the frame rate, i.e., number of images rendered per second, taking a big hit, even halving performance in some cases. This is where DLSS comes in.

By reducing the resolution, i.e., the number of pixels, in the GPU’s processing pipeline and upscaling it using convolutional neural networks, DLSS was able to improve performance by up to 50% in some cases. To power the AI algorithms used by DLSS, NVIDIA also integrated Tensor cores into the RTX cards. Due to the large amount of specialised hardware required to make DLSS work, it has been restricted only to NVIDIA’s 20-series chips and above—with DLSS 3 only being available on 40-series GPUs.

AMD, on the other hand, observed the potential that FSR could have for gamers playing on the low-end of the hardware spectrum. FSR was launched in June 2021, lagging behind DLSS by more than two years. Furthermore, AMD’s offering didn’t even use AI, instead relying on spatial upscaling to achieve the same outcome. With the release of FSR 3, the question remains: Can AMD beat NVIDIA even without the use of deep learning?

How does FSR stack up?

FSR 3.0, which is still in development, aims to bring improvements over existing FSR by introducing frame interpolation. Frame interpolation is the process of synthesising ‘in-between’ frames for two frames, increasing the overall frame rate while maintaining quality. By introducing frame interpolation, AMD is beginning to catch up to NVIDIA whose DLSS 3 uses optical flow interpolation to achieve the same purpose.

The consensus among many hardware reviewers and gamers is that DLSS is generally the superior upscaling technology, in terms of quality, when compared to FSR, even though both of them offer close to the same amount of performance gains. While they both are equally efficient, NVIDIA is closed-source, increasing the barrier for entry for game studios. Moreover, it only runs on NVIDIA hardware, and that too only for GPUs of 20-series and above.

FSR, on the other hand, is a set of spatial upscaling algorithms that can quickly be integrated into any game. The algorithms were also made open source shortly after launch, leading to more game companies integrating FSR than DLSS, even though most games include an option for both. The nature of the algorithms mean that it can be run on any hardware, with even NVIDIA’s 10-series cards being supported by the technology.  

To achieve optimal upscaling results, NVIDIA makes gratuitous use of auto-encoded convolutional neural networks. These networks were built from the ground up for upscaling tasks, and the model is fine-tuned for every game that has support for DLSS. This means that a game has to support DLSS from the development stage, with game devs sharing datasets of images from their game to train the DLSS CNN.

Intel has also thrown its hat in the ring, equipping its Intel ARC GPUs with a technology they have termed Xeon Super Sampling, or XeSS. This technology also uses AI for upscaling, but is only supported by a handful of games. While AMD’s FSR is adopted by 226 titles as of December 2022 and NVIDIA’s is adopted by 210, XeSS is adopted by a measly 20. 

With FSR 3.0’s non-hardware-accelerated upscaling algorithms, there comes a catch due to interpolation. Frame interpolation typically adds latency—a big no-go for gamers. AMD has clarified that FSR 3.0 is still in development, meaning that there is still time for it to enhance its latency capabilities. 

As NVIDIA continues to build its walled garden, it seems that it is moving towards pushing consumers to buy its newer cards to take advantage of DLSS’ latest advancements. AMD, on the other hand, is building FSR to be used with as many games and as many GPUs as possible, with the only gap left to be closed being that of visual fidelity. 

The post DLSS vs. FSR 3.0: Is AI Enough to Give NVIDIA the Edge? appeared first on Analytics India Magazine.