• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

NVIDIA Adds AI Frame Generation Capability to Video Encoding and Decoding, Increase Frame-Rates of Videos

btarunr

Editor & Senior Moderator
Staff member
Joined
Oct 9, 2007
Messages
47,194 (7.56/day)
Location
Hyderabad, India
System Name RBMK-1000
Processor AMD Ryzen 7 5700G
Motherboard ASUS ROG Strix B450-E Gaming
Cooling DeepCool Gammax L240 V2
Memory 2x 8GB G.Skill Sniper X
Video Card(s) Palit GeForce RTX 2080 SUPER GameRock
Storage Western Digital Black NVMe 512GB
Display(s) BenQ 1440p 60 Hz 27-inch
Case Corsair Carbide 100R
Audio Device(s) ASUS SupremeFX S1220A
Power Supply Cooler Master MWE Gold 650W
Mouse ASUS ROG Strix Impact
Keyboard Gamdias Hermes E2
Software Windows 11 Pro
The defining feature of DLSS 3 is AI frame-generation, the ability of NVIDIA GeForce RTX 40-series "Ada" GPUs to predict the next frame to one that's rendered by the GPU, and generate the frame without any involvement of the graphics rendering pipeline. NVIDIA is taking this concept to video encoding, too, letting you increase the frame-rate of your videos through the "magic" of frame generation. NVIDIA Ada GPUs' Optical Flow Accelerator (NVOFA) component can apply the same Optical Flow logic for videos as it does for graphics rendering, predict the next frame, and increase frame-rate through AI generation of that frame. NVIDIA refers to this as Engine-assisted Frame-rate Up Conversion (FRUC).

There's more to FRUC than the "smooth motion" features your TV comes with; NVENC compares two real frames from a video, determines motion vectors, and sets up an optical flow stage, so the generated frames that are interpolated with real frames are accurate. NVIDIA will be releasing FRUC as a library, so it can be integrated with popular content-creation and media-consumption applications on NVIDIA Ada GPUs. It allows people with Ada to create higher frame-rate videos; as well as those with Ada GPUs to consume media at higher frame-rates.



A video presentation by NVIDIA on the video encoding features of Ada follows.

View at TechPowerUp Main Site | Source
 
Joined
Feb 11, 2009
Messages
5,540 (0.96/day)
System Name Cyberline
Processor Intel Core i7 2600k -> 12600k
Motherboard Asus P8P67 LE Rev 3.0 -> Gigabyte Z690 Auros Elite DDR4
Cooling Tuniq Tower 120 -> Custom Watercoolingloop
Memory Corsair (4x2) 8gb 1600mhz -> Crucial (8x2) 16gb 3600mhz
Video Card(s) AMD RX480 -> RX7800XT
Storage Samsung 750 Evo 250gb SSD + WD 1tb x 2 + WD 2tb -> 2tb MVMe SSD
Display(s) Philips 32inch LPF5605H (television) -> Dell S3220DGF
Case antec 600 -> Thermaltake Tenor HTCP case
Audio Device(s) Focusrite 2i4 (USB)
Power Supply Seasonic 620watt 80+ Platinum
Mouse Elecom EX-G
Keyboard Rapoo V700
Software Windows 10 Pro 64bit
actually sounds pretty cool
 
Joined
Dec 30, 2021
Messages
384 (0.37/day)
This is neat, but also if the reason DLSS Frame Generation can't work with Ampere and Turing is because those GPUs don't have powerful enough optical flow engines for real-time frame generation, then what's the excuse for them not supporting it in offline video encoding either? It doesn't need to be real-time.
 
Joined
Feb 23, 2019
Messages
6,036 (2.89/day)
Location
Poland
Processor Ryzen 7 5800X3D
Motherboard Gigabyte X570 Aorus Elite
Cooling Thermalright Phantom Spirit 120 SE
Memory 2x16 GB Crucial Ballistix 3600 CL16 Rev E @ 3800 CL16
Video Card(s) RTX3080 Ti FE
Storage SX8200 Pro 1 TB, Plextor M6Pro 256 GB, WD Blue 2TB
Display(s) LG 34GN850P-B
Case SilverStone Primera PM01 RGB
Audio Device(s) SoundBlaster G6 | Fidelio X2 | Sennheiser 6XX
Power Supply SeaSonic Focus Plus Gold 750W
Mouse Endgame Gear XM1R
Keyboard Wooting Two HE
Can't wait to watch Citizen Kane in 120fps :roll:
 
D

Deleted member 185088

Guest
This is neat, but also if the reason DLSS Frame Generation can't work with Ampere and Turing is because those GPUs don't have powerful enough optical flow engines for real-time frame generation, then what's the excuse for them not supporting it in offline video encoding either? It doesn't need to be real-time.
The whole reason is force people to buy RTX4xxx, people are as blind towards nVidia as Apple, the only difference nVidia's products are both overpriced and good.
 

dgianstefani

TPU Proofreader
Staff member
Joined
Dec 29, 2017
Messages
5,000 (1.99/day)
Location
Swansea, Wales
System Name Silent
Processor Ryzen 7800X3D @ 5.15ghz BCLK OC, TG AM5 High Performance Heatspreader
Motherboard ASUS ROG Strix X670E-I, chipset fans replaced with Noctua A14x25 G2
Cooling Optimus Block, HWLabs Copper 240/40 + 240/30, D5/Res, 4x Noctua A12x25, 1x A14G2, Mayhems Ultra Pure
Memory 32 GB Dominator Platinum 6150 MT 26-36-36-48, 56.6ns AIDA, 2050 FCLK, 160 ns tRFC, active cooled
Video Card(s) RTX 3080 Ti Founders Edition, Conductonaut Extreme, 18 W/mK MinusPad Extreme, Corsair XG7 Waterblock
Storage Intel Optane DC P1600X 118 GB, Samsung 990 Pro 2 TB
Display(s) 32" 240 Hz 1440p Samsung G7, 31.5" 165 Hz 1440p LG NanoIPS Ultragear, MX900 dual gas VESA mount
Case Sliger SM570 CNC Aluminium 13-Litre, 3D printed feet, custom front, LINKUP Ultra PCIe 4.0 x16 white
Audio Device(s) Audeze Maxwell Ultraviolet w/upgrade pads & LCD headband, Galaxy Buds 3 Pro, Razer Nommo Pro
Power Supply SF750 Plat, full transparent custom cables, Sentinel Pro 1500 Online Double Conversion UPS w/Noctua
Mouse Razer Viper Pro V2 8 KHz Mercury White w/Tiger Ice Skates & Pulsar Supergrip tape
Keyboard Wooting 60HE+ module, TOFU-R CNC Alu/Brass, SS Prismcaps W+Jellykey, LekkerV2 mod, TLabs Leath/Suede
Software Windows 11 IoT Enterprise LTSC 24H2
Benchmark Scores Legendary
The whole reason is force people to buy RTX4xxx, people are as blind towards nVidia as Apple, the only difference nVidia's products are both overpriced and good.
I suspect the market for people who would reencode a saved video, before watching it locally is rather small.

The market for real time reencoding however is much, much higher. Since previous gen can't do real time, at all, that's probably the reason, along with obviously marketting RTX4xxx.
 
Joined
Sep 1, 2022
Messages
487 (0.61/day)
System Name Firestarter
Processor 7950X
Motherboard X670E Steel Legend
Cooling LF 2 420
Memory 4x16 G.Skill X5 6000@CL36
Video Card(s) RTX Gigabutt 4090 Gaming OC
Storage SSDS: OS: 2TB P41 Plat, 4TB SN850X, 1TB SN770. Raid 5 HDDS: 4x4TB WD Red Nas 2.0 HDDs, 1TB ext HDD.
Display(s) 42C3PUA, some dinky TN 10.1 inch display.
Case Fractal Torrent
Audio Device(s) PC38X
Power Supply GF3 TT Premium 850W
Mouse Razer Basilisk V3 Pro
Keyboard Steel Series Apex Pro
VR HMD Pimax Crystal with Index controllers
I suspect the market for people who would reencode a saved video, before watching it locally is rather small.

The market for real time reencoding however is much, much higher. Since previous gen can't do real time, at all, that's probably the reason, along with obviously marketting RTX4xxx.
Yeah, video makers might find this useful, I personally use Dainapp for getting rid of lag spikes in the footage.
 
Joined
Feb 11, 2009
Messages
5,540 (0.96/day)
System Name Cyberline
Processor Intel Core i7 2600k -> 12600k
Motherboard Asus P8P67 LE Rev 3.0 -> Gigabyte Z690 Auros Elite DDR4
Cooling Tuniq Tower 120 -> Custom Watercoolingloop
Memory Corsair (4x2) 8gb 1600mhz -> Crucial (8x2) 16gb 3600mhz
Video Card(s) AMD RX480 -> RX7800XT
Storage Samsung 750 Evo 250gb SSD + WD 1tb x 2 + WD 2tb -> 2tb MVMe SSD
Display(s) Philips 32inch LPF5605H (television) -> Dell S3220DGF
Case antec 600 -> Thermaltake Tenor HTCP case
Audio Device(s) Focusrite 2i4 (USB)
Power Supply Seasonic 620watt 80+ Platinum
Mouse Elecom EX-G
Keyboard Rapoo V700
Software Windows 10 Pro 64bit
Joined
Feb 1, 2013
Messages
1,264 (0.29/day)
System Name Gentoo64 /w Cold Coffee
Processor 9900K 5.2GHz @1.312v
Motherboard MXI APEX
Cooling Raystorm Pro + 1260mm Super Nova
Memory 2x16GB TridentZ 4000-14-14-28-2T @1.6v
Video Card(s) RTX 4090 LiquidX Barrow 3015MHz @1.1v
Storage 660P 1TB, 860 QVO 2TB
Display(s) LG C1 + Predator XB1 QHD
Case Open Benchtable V2
Audio Device(s) SB X-Fi
Power Supply MSI A1000G
Mouse G502
Keyboard G815
Software Gentoo/Windows 10
Benchmark Scores Always only ever very fast
Svp + rife or optical flow. Supports back to Turing.
 

OneMoar

There is Always Moar
Joined
Apr 9, 2010
Messages
8,794 (1.65/day)
Location
Rochester area
System Name RPC MK2.5
Processor Ryzen 5800x
Motherboard Gigabyte Aorus Pro V2
Cooling Thermalright Phantom Spirit SE
Memory CL16 BL2K16G36C16U4RL 3600 1:1 micron e-die
Video Card(s) GIGABYTE RTX 3070 Ti GAMING OC
Storage Nextorage NE1N 2TB ADATA SX8200PRO NVME 512GB, Intel 545s 500GBSSD, ADATA SU800 SSD, 3TB Spinner
Display(s) LG Ultra Gear 32 1440p 165hz Dell 1440p 75hz
Case Phanteks P300 /w 300A front panel conversion
Audio Device(s) onboard
Power Supply SeaSonic Focus+ Platinum 750W
Mouse Kone burst Pro
Keyboard SteelSeries Apex 7
Software Windows 11 +startisallback
this is not new projects like svp and RIFF have been doing this for years with varying results riff can use vulcan or cuda but I don't think its as fast as what nvidia is promising
also trying to use this in a multiplayer environment is insanity
 
Joined
Dec 19, 2008
Messages
305 (0.05/day)
Location
WA, USA
System Name Desktop
Processor AMD Ryzen 5950X
Motherboard ASUS Strix B450-I
Cooling be quiet! Dark Rock TF 2
Memory 32GB DDR4 3600
Video Card(s) AMD RX 6800
Storage 480GB MyDigitalSSD NVME
Display(s) AOC CU34G2X
Power Supply 850w
Mouse Razer Basilisk V3
Keyboard Steelseries Apex 5
So that feature on TV's that every movie director and cinephile tells you to disable is now on nvidia. Neat.
 
Joined
Feb 23, 2008
Messages
1,064 (0.17/day)
Location
Montreal
System Name Aryzen / Sairikiki / Tesseract
Processor 5800x / i7 920@3.73 / 5800x
Motherboard Steel Legend B450M / GB EX58-UDP4 / Steel Legend B550M
Cooling Mugen 5 / Pure Rock / Glacier One 240
Memory Corsair Something 16 / Corsair Something 12 / G.Skill 32
Video Card(s) AMD 6800XT / AMD 6750XT / Sapphire 7800XT
Storage Way too many drives...
Display(s) LG 332GP850-B / Sony w800b / Sony X90J
Case EVOLV X / Carbide 540 / Carbide 280x
Audio Device(s) SB ZxR + GSP 500 / board / Denon X1700h + ELAC Uni-Fi 2 + Senn 6XX
Power Supply Seasonic PRIME GX-750 / Corsair HX750 / Seasonic Focus PX-650
Mouse G700 / none / G602
Keyboard G910
Software w11 64
Benchmark Scores I don't play benchmarks...
So, essentially is "Soap Opera" on steroids?

Curious, how many of you have been leaving the image interpolation function on your TVs for the past decade?
 
Joined
Jan 14, 2019
Messages
12,334 (5.80/day)
Location
Midlands, UK
System Name Nebulon B
Processor AMD Ryzen 7 7800X3D
Motherboard MSi PRO B650M-A WiFi
Cooling be quiet! Dark Rock 4
Memory 2x 24 GB Corsair Vengeance DDR5-4800
Video Card(s) AMD Radeon RX 6750 XT 12 GB
Storage 2 TB Corsair MP600 GS, 2 TB Corsair MP600 R2
Display(s) Dell S3422DWG, 7" Waveshare touchscreen
Case Kolink Citadel Mesh black
Audio Device(s) Logitech Z333 2.1 speakers, AKG Y50 headphones
Power Supply Seasonic Prime GX-750
Mouse Logitech MX Master 2S
Keyboard Logitech G413 SE
Software Bazzite (Fedora Linux) KDE
This is neat, but also if the reason DLSS Frame Generation can't work with Ampere and Turing is because those GPUs don't have powerful enough optical flow engines for real-time frame generation, then what's the excuse for them not supporting it in offline video encoding either? It doesn't need to be real-time.
Also, what does "not powerful enough" mean? Not powerful enough to render a 3D scene? Fine, then give us the support in 2D video decoding. If Turing's OFA is too weak for that too, then why does it exist?

I can smell a big "actually, Turing can have it too" announcement 6-12 months down the line when all the poor souls are already paying their loans they took out to switch to Ada.

Edit: I can also smell AMD developing a similar technology and enabling it for everything above the RX 580.
 
Top