How Ray Reconstruction Works

These two slides are from NVIDIA's original DLSS 3.5 presentation, and they still explain the idea very well. The first shows the traditional pipeline, the second shows why it's a problem.

In a classic ray tracing pipeline, the engine first builds geometry and materials without shading, and then uses that to construct the BVH acceleration structure, and then casts a limited number of rays into the scene. Because there are far fewer rays than pixels, the raw output is extremely noisy, so a denoiser is brought in to turn those missing samples into something that looks like a proper image. With upscaling enabled, that denoiser runs at the lower render resolution and has no idea what the final output resolution will be. The upscaler that runs afterward, in turn, has no idea what a ray is—all it sees is the denoiser's pixel output—a flat image, and every bit of ray tracing information is lost.

Ray Reconstruction fixed that by making denoising and upscaling a single AI-powered pass that receives the noisy ray traced samples, and supporting values to make an informed decision and write directly into the final output resolution.

What the Model Actually Receives

NVIDIA's slides show three input images labeled "Color / Motion vector / Ray samples," and add a fourth black-and-white image for the DLSS 4.5 diagram, without ever describing what that data is.

The reality is completely different. A game with ray reconstruction support actually exposes up to 20+ separate buffers to the DLSS algorithm, here's a list:

• Materials: diffuse albedo, specular albedo, normals, roughness, reflected albedo, emissive, indirect albedo.
• Ray data: diffuse and specular hit distance, diffuse and specular ray direction.
• Transparency: transparency layer, its motion vectors and opacity, plus the color buffers from before and after transparency is composited.
• Guides: a screen-space subsurface scattering guide and a depth-of-field guide.
• Masks: a responsivity mask, a disocclusion mask and a ghost-reduction mask.
• Misc: depth, alpha, pre-exposure, exposure scale, specular motion vectors.

Of course, it's not unreasonable for NVIDIA to simplify things a bit when talking to a non-game developer audience, it would still be nice if their briefing materials could actually contain the correct information, so we can handle the job of translating it for our audience.

What's important is that most of these are optional, which is very important for adoption rates. A game studio can hammer out a first build with just the core G-buffer set in a few days (albedo, normals, roughness, depth, motion vectors, the noisy radiance) and they have a working build. Once everything is live, they can iteratively improve based on feedback from QA, NVIDIA and gamers. That keeps the cost of the integration low while leaving headroom for studios with bigger budgets.

What's New in the 2nd Generation Transformer Model

DLSS 4 already moved Ray Reconstruction from a convolutional neural network to a transformer model. DLSS 4.5 keeps the transformer architecture, but scales it up with better training and new data. I'm also sure that NVIDIA identified specific weaknesses and gave those some extra love (like text rendering).

NVIDIA's documentation lists the following, which isn't much:

• Efficient Denoiser: The new model delivers 35% more compute capability, and processes 20% more parameters, while maintaining similar performance to the previous model.
• Enhanced Super Resolution: Building upon the advances from DLSS 4.5 Super Resolution, the model has deeper spatial awareness across every scene, and more intelligently uses game engine pixel sampling and motion data. The result is improved lighting accuracy, better temporal stability and clearer motion in ray-traced and path-traced content.
• Expanded Training Dataset: Trained on a larger dataset, the new model is even better at image reconstruction. This intelligence gives the model even better awareness to pick the most accurate engine data to reconstruct scenes closer to ground truth.
• Finer Developer Control: The new model provides developers with finer control for temporal accumulation, providing precise tuning of model response for even better image quality.

Spot the Difference

Here are the two diagrams, DLSS 4 Ray Reconstruction on the left and DLSS 4.5 Ray Reconstruction on the right. Open them and arrow left and right between the two, because at a glance they appear to be the same image.


They are nearly identical indeed. No changes to the pipeline: the same inputs feed into a single transformer block, which writes to the output, and the same breakout box below shows denoising and Super Resolution happening together inside that one model rather than as two separate passes. Same arrows, same structure, same everything. Two things actually change.

The input stack gains a fourth plane. DLSS 4 shows three stacked input images, DLSS 4.5 shows four. I briefly mentioned that above. The caption underneath changes too, but not in a helpful way: DLSS 4 labels the stack "Color / Motion vector / Ray samples," which at least tells you what you're looking at, while DLSS 4.5 replaces that with "New fine-grain control for temporal accumulation." So the one label that identified the inputs has been swapped out for a marketing line, and the new fourth plane is never named, but I suspect it's one of the more exotic masks above.

The two boxes at the bottom gain adjectives. "AI Denoiser" becomes "Efficient AI Denoiser," "AI Super Resolution" becomes "Enhanced Super Resolution," and each picks up a caption restating the bullet points above. No new stages, no reordering, no change to what connects to what.

What We Think This Means

The fact that NVIDIA shows the same diagram, while they had every opportunity to show a restructured pipeline and didn't, I'd say this confirms that the same unified Denise-and-upscale design introduced with DLSS 4 is used, but made larger, retrained on more data, and designed as a replacement drop-in. It is a new model, not a new architecture, which isn't unreasonable given the ".5" suffix on DLSS 4.5.

The fourth input plane can't be an essential input that the model requires to function. Otherwise, the NVIDIA App override wouldn't be able to upgrade all the existing shipping games. The override cannot magically make games send new data to the DLSS algorithm. I rather suspect that these are the optional inputs, and maybe NVIDIA changed the new algorithm to make better use of those inputs, or differently.