Friday, March 20th 2026
NVIDIA DLSS 5 Takes 2D Frame and Motion Vectors as Input
The public is still learning more about DLSS 5 technology, and NVIDIA is gradually revealing additional details. In a conversation between YouTuber Daniel Owen and NVIDIA's GeForce Evangelist Jacob Freeman, we discover that DLSS 5 essentially takes 2D frames and motion vectors as input, applying its generative AI model to output frames in a 2D context rather than 3D. This means that while the model is trained on materials that might appear 3D, its actual work is based on 2D imagery. It identifies each frame's motion vectors and anchors the model to them, operating in a 2D space instead of a full 3D environment. This approach to handling graphics is much more computationally efficient, as achieving complete photorealism in 3D would require more GPU power than is currently available. No underlying geometry is changed. Instead, 2D images and motion vectors serve as input, a generative AI model processes textures and applies scene photorealism, resulting in the final output that NVIDIA promotes as DLSS 5.Below, you can see the complete Q&A exchange between Daniel Owen and Jacob Freeman, which was nicely compiled by VideoCardz. Numbers represent questions from Daniel Owen, while answers are provided by NVIDIA's Jacob Freeman.
Sources:
Daniel Owen on YouTube, via VideoCardz
1: My biggest question is whether or not this model is effectively taking a single 2D frame as an input (with motion vectors) to create the output frame? This then relates to some clarifying questions about the information you provided, which I ask below:
Answer: Yes, DLSS 5 takes a 2D frame plus motion vectors as input.
2: You say the underlying geometry and textures are unchanged. Is the model actually aware of things like 3D geometry, 3D depth, etc, or is it essentially just looking at a 2D image and then reinterpreting it similar to taking a screenshot of a game and asking an AI model to enhance the screenshot to make it more realistic?
Answer: DLSS 5 is trained end to end to understand complex scene semantics such as characters, hair, fabric and translucent skin, along with environmental lighting conditions like front-lit, back-lit or overcast, all by analyzing a single frame.
4. You say that "With DLSS 5 the underlying geometry and textures are unchanged." However, in one of the publicly available examples, it looks like a character grew hair in a place where it didn't exist before. This seems to be a change to the model geometry and/or textures rather than just an enhancement of the lighting. In other words, it seems like an AI image generator reinterpreting the character, which can cause actual details to be changed. It also appears like some characters gain makeup that the original model did not seem to be wearing, which seems like changing the underlying texture.
Answer: The underlying geometry is unchanged. Also worth mentioning this is a very early preview of the tech.
4. You say it enhances PBR properties on materials. Is it aware of the PBR properties by reading them from the game engine in some way, or is it just "looking" at the output image and inferring them. In other words, it is my understanding that game artists traditionally tell a shader what specific parts of a 3D model look like by giving it properties like metallic, roughness, normal maps (to simulate "bumpiness"), etc. Is DLSS 5 taking those artist created values as inputs, or is it inferring them from something more like the final output image.
Answer: DLSS 5 only takes the rendered frame and motion vectors as inputs. Materials are inferred from the rendered frame.
5. When you describe the tools the developers have to control the model, it does not sound like there is actually a lot of control over the DLSS5 model's interpretation of the artistic intent of a scene beyond color grading (or turning DLSS 5 "off" or "down" on some or all elements of the scene). For example, in the opening scene of Resident Evil Requiem, Grace is a character with a lot of trauma, who is reluctantly on her way to investigate a murder in the same location where her mother was murdered several years earlier. With DLSS 5 Off, I get more of that impression of the character. With DLSS 5 on, while she does seem to have more photo realistic lighting and skin, it also looks like she is wearing makeup that was not present in the original version. This gives off a very different impression of the character and emotional impact of the scene. Can developers actually control anything about the output image besides color grading? If they see the DLSS5 output and want to tweak it, like asking it not to apply makeup to the character, are they able to do so?
Answer: Developers will have detailed controls such as intensity and color grading. Artists can use these controls to adjust blending, contrast, saturation and gamma, and determine where and how enhancements are applied to maintain the games' unique aesthetic. Developers can also mask specific objects or areas to be excluded from enhancement. We continue to talk to developers to understand all the ways they would like to control the technology. Ultimately, we see DLSS 5 as a tool for them to achieve their artistic vision, rather than be limited by the capabilities of traditional real-time rendering.
6. Is this model limited to screen space? Or does it have any awareness of the environment outside of what is visible in an individual game frame?
Answer: DLSS 5 only takes the rendered frame and motion vectors as inputs.
7. With the introduction of DirectML to DirectX allowing for hardware agnostic acceleration of ML tasks in games, does NVIDIA plan to continue with vendor locked versions of their machine learning based game technology like DLSS 5? Or will NVIDIA allow these models to run through DirectML on any hardware?
Answer: We have nothing to announce on this one at this time.

30 Comments on NVIDIA DLSS 5 Takes 2D Frame and Motion Vectors as Input
Why else would it create lights where they don't exist? It's not because it's interpreting the materials better than rasterisation could (because it couldn't do that regardless), but because it's just making it up.
There are already AI models that do :
2D picture -> 3D meshes + texture.
So actually making an AI that create a final 2D image (a frame) using actual meshes, textures, position and light info should be possible. Maybe not now, but if we want neural rendering to produce result, we need to use all data available in their full granularity not using the aggregate that is already the final frame and slap over it some lazy generative AI.
Edit : Nvidia used to be the power that move forward computer graphics, but honestly for the last 10+ years, they have been more and more lazy, innovating but not in the cleverest way anymore. They are lucky that AMD is completely incapable of innovating by themself.
We can argue about it, we can disagree about it, but that's about all there is to it.
like microsoft recall. really copilot in general.
PhysX was enough to convince developers to ship games with crippled physics without it, making them look noticeably worse on non‑Nvidia hardware. It failed in the long run because Nvidia wasn’t a monopoly back then.
GameWorks libraries made games run worse on anything other than Nvidia hardware, but AMD had already secured the two major consoles by that time, so the damage was limited.
Ray tracing and path tracing probably make programmers lazier with traditional lighting, while also killing performance and pushing many players toward more expensive options. Fortunately, people aren’t completely sold on this yet, but that could change in the future.
DLSS and upscaling in general are a big reason we get so many unoptimized games. With Nvidia’s market share being over 90%, the need for DLSS is treated like a non‑issue, and frame generation —especially multi frame generation— can only guarantee that games become even less optimized. With so many people buying high refresh rate monitors, cards that don’t offer multi frame generation may start to look like a bad match for those displays.
And now we have this DLSS 5 “photorealism.” The current backlash against DLSS 5 is actually a good sign, but Nvidia is a very strong monopoly. I expect later DLSS versions doing similar tricks to receive much more positive coverage from the tech press. I wouldn’t be surprised if faces in games start looking worse, and this “photorealism” that DLSS 5 generates gradually becomes a necessity just to get graphics that look more like 2030 than 2010.
In any case, Nvidia is trying to
- eliminate competition by pushing proprietary features that only work on Nvidia hardware
- lock gamers into its ecosystem
- push gamers toward more frequent and often unnecessary upgrades
Even G‑Sync, which I haven’t really mentioned here, drove monitor prices up significantly. It was a welcome feature, but once again it was designed in a way that funneled money to Nvidia while locking gamers in and keeping competitors out.
it's light rays currently for ray tracing cores That's slowing these cards down, because these cards are hybrids between both rasterization & Ray tracing/path tracing & A.I performance.
If the cards didn't do both they the would be faster at both separately while being two different non-compatible cards. Then trying force it through the same rend out put.
The cards are being jack of all trades while being masters of nothing.
www.digitalfoundry.net/features/nvidias-new-dlss-5-brings-photo-realistic-lighting-to-rtx-50-series
Thanks to "pgcrt" who made this video and shader/addon.
for the nerd´s, here are teh HW specs on which this ran:if u want to know more, feel free to join our ReShade Discord <3
-------------------------------------------------------------------------------------------------------------------
and while we are at it, i guess i also say what i think about DLSS 5.
there is nothing special, or "a huge achivement" about DLSS5. the only thing they did is to create a realtime, scene-aware, yassify filter that uses motion vectors and masking so teh slop stays where it is supposed to. it is not even aware of DEPTH.
it has nothing to do with real game/engine rendering or the render pipeline. it doesnt use 3d, it doesnt use a depth buffer or normals, it doesnt know world space (it uses 2d screenspace - the image itself). it is a very very lazy way of advancing game graphics. honestly, as someone who understands a lot of teh stuff we do, i kinda feel offended by NVIDIA. this feels more like a scam from a Conartist.
the sad thing is that most people dont even understand how lazy this is. (no wonder cause jensen used some really bad corpo marketing language to make it sound way bigger then it actually is, but if u understand how stuff works, you can see through this bullshit.)
it uses teh 2d image that u see on screen, motion vectors as mentioned above etc, and then, in realtime, just slops a beautify filter on it which is trained on a lot of images and then gets averaged, so u get the samish look as in all ur other ai beautify filters.
and the "config options" artists have to use this slop makes it even more clear..
does that not sound like options u get when you use generative Ai to beautify ur image? (img2img) and not some tools that are integrated into teh pipeline/engine etc
heck, even ReShade and some of its addons/shaders know more about the game then dlss5.
-------------------------------------------------------------------------------------------------------------------
all this totally reminds me of the scooby doo meme where they pull the mask off the bad guy.
anyhow, have a wonderful evening :p
DLSS5 needs some(a lot of?) refinement by human beings to reach an optimal state of usefulness and visual appeal.
Like DLSS when it first released, it has good potential IF done right.
Look, DLSS 5 looks like shit, I think most would agree with this, but no need for complete delusion in discussing it.Yes, theoretically. The problem is that the emphasis is on “a lot of manual refinement” which kind of makes the whole thing near-pointless. Plus, the subject of the article itself, it being 2D frame based, makes it actually rather limited as far as said manual refinement goes. At the end of the day, it IS a glorified AI filter.
So who knows at all...
Also...
PS: I wonder if motherboards with more than one PCIe x16 slot, will make a comeback.
Also I must add that the data set use by Nvidia to train their model was probably bad, as "freely available" on social media bad. It was obvious by looking at yassified Grace face, but even exterior have that badly done overprocessed "HDR photography" look. If we look at the mountain or tree in AC Shadow, we can see the typical darkening next to where the sky start that is a clue of overcook hdr photo, and those darkening weren't present in the dlss off image...
Nvidia trying to market many different techs under one brand.
DLSS is the most successful part of their AI tech used in gaming, so they now just putting everything new under its umbrella.
We need each tech showcased by itself.
So here is super sampling on/off (the actual DLSS)
Here is our AA replacement on /off (DLAA)
Here is frame gen on/off (DLFG)
Here is our facial reconstruction (contrast lighting filter) on/off (DLFC)
Instead its here is everything on at the same time.
The way they made 3dmark force FG to be enabled when DLSS is turned on in latest version, and tried to push reviewers to only bench with FG turned on.