NVIDIA Releases VRWorks Audio and 360 Video SDKs at GTC

[Raevenlord]

Further planting its roots on the VR SDK and development field, NVIDIA has just announced availability of two more SDK packages, for their VRWorks Audio and 360 Video suites. Now a part of NVIDIA's VRWorks suite of VR solutions, the VRWorks Audio SDK provides real-time ray tracing of audio in virtual environments, and is supported in Epic's Unreal Engine 4 (here's hoping this solution, or other solutions similar to it, address the problem of today's game audio.) The VRWorks 360 Video SDK, on the other hand, may be less interesting for graphics enthusiasts, in that it addresses the complex challenge of real-time video stitching.

Traditional VR audio ( and gaming audio, for that matter) provide an accurate 3D position of the audio source within a virtual environment. However, as it is handled today, sound is processed with little regard to anything else but the location of the source. With VRWorks Audio, NVIDIA brings to the table considerations for the dimensions and material properties of the physical environment, helping to create a truly immersive environment by modeling sound propagation phenomena such as reflection, refraction and diffraction. This is to be done in real time, at a GPU level. This work leverages NVIDIA's OptiX ray-tracing technology, which allows VRWorks Audio to trace the path of sound in real time, delivering physically accurate audio that reflects the size, shape and material properties of the virtual environment.





NVIDIA says that this is, at least for now, the only hardware-accelerated and path-traced audio solution that creates a complete acoustic image of the environment in real time without requiring any "pre-baked" knowledge of the scene. As the scene is loaded by the application, the acoustic model is built and updated on the fly, with audio effect filters being generated and applied on the sound source waveforms. The software release consists of a set of C-APIs for integration into any engine or application, and an integration for Epic's Unreal Engine 4 that's now available on GitHub.

As to its VRWorks 360 Video SDK, it enables real-time 4K 360 video capture, stitching and streaming. Regarding this feature, Kinson Loo, CEO of Z CAM, said that "The fact that NVIDIA manages to stitch 4K 360 stereoscopic video in real time, making livestreaming possible, changes the production pipeline and enables entirely new use cases in VR."

View at TechPowerUp Main Site

 

[xkm1948]

This is true dedication to VR!

 

[birdie]

When will this VR fad end? I'm freaking fed up with it.

 

[P4-630]

When will this VR fad end? I'm freaking fed up with it.

It just started lol!!......

 

[xkm1948]

When will this VR fad end? I'm freaking fed up with it.

It just started lol!!......

Buckle up kittens, your so hated "VR fad" is gonna be around for a lot longer than you can imagine.

 

[birdie]

Buckle up kittens, your so hated "VR fad" is gonna be around for a lot longer than you can imagine.

We've had failed attempts at VR several times already. This one will wither out as well. Very few actually want to wear clunky BS on their heads.

 

[xkm1948]

We've had failed attempts at VR several times already. This one will wither out as well. Very few actually want to wear clunky BS on their heads.

Yeah hate all you want kitten. Only time will tell whether you are right or wrong this time.

 

[alucasa]

FYI, it's a birdie, not kitten. A huge difference in my tiny mind.

 

[xkm1948]

FYI, it's a birdie, not kitten. A huge difference in my tiny mind.

Nah I prefer to call them kittens, young and cute with their lovely biased heads.

 

[Hood]

Nah I prefer to call them kittens, young and cute with their lovely biased heads.

Not Fair! I thought this was a safe space, where all my delusions can thrive and fester - I,m leaving, and taking my vast storehouse of computer knowledge and conspiracy theories with me! So there!

 

[alucasa]

Nooo, don't leave, Robin Hood!

 

[kn00tcn]

When will this VR fad end? I'm freaking fed up with it.

why are you seemingly angry at something that doesnt affect you? or are you implying that resources are spent away from cpu/gpu advancements or something? (at a time when most games run great & dont require constant high end upgrades, VR comes in demanding low latency high framerate high resolution multiple camera renders...)

have you tried any headset? it's not clunky if it's light & fits well, if backpack computers are becoming a thing thanks to the performance we get per watt by now

did old failed VR look good? did it have positional tracking? no, old VR sucked

let me show you what i tried a few days ago

it was a project for museums where you can walk through an upscaled CT scanned archaeological object, an object that will be damaged if it was physically opened, but turning it into a 3d render means we can examine it without any worries

therefore it's naive & offensive to think an entire platform, a user interface, a set of concepts has anything to do with a flood of pointless pew pew games on steam, as if this exact conversation didnt happen over & over again with smartphones, social networks, 2d games, movies, or books

so if you have an actual idea of what VR should be, start making it, unreal & unity are free... or tell us the idea so someone else can make it

 

[Prima.Vera]

Why do you need only a GPU for this. Why not using an API all users can have access, like OpenAL or DS3D (or something).
My 5 years old Creative card is more than up for the task.

 

[RejZoR]

1999 called and it wants it's Aureal A3D back. Then, 2005 called and wanted EAX 5.0 back as well.

We've had audio tracing with environmental effects even back then and after sacking DirectSound 3D and all extensions for it, fast forwarding 2 decades and we're again at point ZERO, reinventing everything we already had ages ago, AGAIN. Sometimes I just don't understand the industry. We already had jaw dropping audio with breathtaking 3D positioning in games and all of it got scraped with dumb Microsoft screwing it all with garbage software audio layer. Which hasn't improved or changed since Vista.

Creative is imo still the king for audio, as much as they can even do on top of a "clueless" audio API. The surround softens the transitions better between audio channels which makes audio tracking better, but they still can't do it as well as if the game actually told the audio API "hey, the sound is coming from this location exactly, project that into speakers/headphones".

People have no clue how far audio in games has regressed since sacking of hardwre accelerated API's...

 

[kn00tcn]

1999 called and it wants it's Aureal A3D back. Then, 2005 called and wanted EAX 5.0 back as well.

We've had audio tracing with environmental effects even back then and after sacking DirectSound 3D and all extensions for it, fast forwarding 2 decades and we're again at point ZERO, reinventing everything we already had ages ago, AGAIN. Sometimes I just don't understand the industry. We already had jaw dropping audio with breathtaking 3D positioning in games and all of it got scraped with dumb Microsoft screwing it all with garbage software audio layer. Which hasn't improved or changed since Vista.

Creative is imo still the king for audio, as much as they can even do on top of a "clueless" audio API. The surround softens the transitions better between audio channels which makes audio tracking better, but they still can't do it as well as if the game actually told the audio API "hey, the sound is coming from this location exactly, project that into speakers/headphones".

People have no clue how far audio in games has regressed since sacking of hardwre accelerated API's...

i'd like to hear what old games sounded like, because this isnt about simple positional audio

"considerations for the dimensions and material properties of the physical environment, helping to create a truly immersive environment by modeling sound propagation phenomena such as reflection, refraction and diffraction" as someone who has produced music, these material effects are too demanding to have been simulated in realtime over 15 years ago (but yes there are shortcuts that could be done instead, games are all about shortcuts & i do like sounding good enough rather than perfectly accurate... riddick dark athena & killzone 2/3 come to mind as having nice sounding fake reflection or occlusion on last gen consoles, the same way an environment map is a fake reflection that looks fine in most situations)