Thursday, September 22nd 2022
NVIDIA Adds AI Frame Generation Capability to Video Encoding and Decoding, Increase Frame-Rates of Videos
The defining feature of DLSS 3 is AI frame-generation, the ability of NVIDIA GeForce RTX 40-series "Ada" GPUs to predict the next frame to one that's rendered by the GPU, and generate the frame without any involvement of the graphics rendering pipeline. NVIDIA is taking this concept to video encoding, too, letting you increase the frame-rate of your videos through the "magic" of frame generation. NVIDIA Ada GPUs' Optical Flow Accelerator (NVOFA) component can apply the same Optical Flow logic for videos as it does for graphics rendering, predict the next frame, and increase frame-rate through AI generation of that frame. NVIDIA refers to this as Engine-assisted Frame-rate Up Conversion (FRUC).
There's more to FRUC than the "smooth motion" features your TV comes with; NVENC compares two real frames from a video, determines motion vectors, and sets up an optical flow stage, so the generated frames that are interpolated with real frames are accurate. NVIDIA will be releasing FRUC as a library, so it can be integrated with popular content-creation and media-consumption applications on NVIDIA Ada GPUs. It allows people with Ada to create higher frame-rate videos; as well as those with Ada GPUs to consume media at higher frame-rates.A video presentation by NVIDIA on the video encoding features of Ada follows.
Source:
NVIDIA
There's more to FRUC than the "smooth motion" features your TV comes with; NVENC compares two real frames from a video, determines motion vectors, and sets up an optical flow stage, so the generated frames that are interpolated with real frames are accurate. NVIDIA will be releasing FRUC as a library, so it can be integrated with popular content-creation and media-consumption applications on NVIDIA Ada GPUs. It allows people with Ada to create higher frame-rate videos; as well as those with Ada GPUs to consume media at higher frame-rates.A video presentation by NVIDIA on the video encoding features of Ada follows.
13 Comments on NVIDIA Adds AI Frame Generation Capability to Video Encoding and Decoding, Increase Frame-Rates of Videos
The market for real time reencoding however is much, much higher. Since previous gen can't do real time, at all, that's probably the reason, along with obviously marketting RTX4xxx.
also trying to use this in a multiplayer environment is insanity
Curious, how many of you have been leaving the image interpolation function on your TVs for the past decade?
I can smell a big "actually, Turing can have it too" announcement 6-12 months down the line when all the poor souls are already paying their loans they took out to switch to Ada.
Edit: I can also smell AMD developing a similar technology and enabling it for everything above the RX 580.