Wednesday, September 24th 2025
NVIDIA Wants Everyone to Own a Digital Avatar with Open-Source Audio2Face Animation Model
NVIDIA is today open-sourcing its Audio2Face 3D animation model, allowing everyone to become the owner of their own digital avatar. The company has trained its Audio2Face technology to produce characters that are true to life, utilizing generative video models for facial expressions, text-to-speech models for human output, and large language models for actual conversations, all combined in a single pipeline to create a digital avatar. All of these stages are fed by analyzing acoustic input, which features characteristics such as intonation and phonemes, and are then translated into a stream of animation data. That data is later mapped to facial expressions, where the mood of the avatar can be interpreted simply by looking at it. The entire data pipeline can be either an offline scripted content or you can feed the model using live data for real-time lip-synching and facial expressions.
Additionally, NVIDIA is open-sourcing the entire Audio2Face stack, which includes the model itself and the SDK, so everyone, from game developers to home enthusiasts, can use it in any possible way. NVIDIA is now open-sourcing the entire training framework, allowing anyone to create their own avatar version with customized features. The company even provided Autodesk Maya and Unreal Engine plugins, allowing 3D workloads to integrate this generative model immediately.Game developers are increasingly treating Audio2Face as a production-ready shortcut to believable facial performance—a tool that automatically converts dialogue into expressive, multilingual lip-sync and facial animation, allowing artists to skip tedious frame-by-frame work and focus on storytelling. Studios from toolmakers like Reallusion, as well as developers such as Survios and The Farm 51, say it speeds up pipelines, frees up animators for higher-value polish, and makes previously impractical ideas achievable, all while keeping characters feeling authentic and emotionally grounded. See how you can get started here. Check out the Audio2Face model in action below:
Additionally, NVIDIA is open-sourcing the entire Audio2Face stack, which includes the model itself and the SDK, so everyone, from game developers to home enthusiasts, can use it in any possible way. NVIDIA is now open-sourcing the entire training framework, allowing anyone to create their own avatar version with customized features. The company even provided Autodesk Maya and Unreal Engine plugins, allowing 3D workloads to integrate this generative model immediately.Game developers are increasingly treating Audio2Face as a production-ready shortcut to believable facial performance—a tool that automatically converts dialogue into expressive, multilingual lip-sync and facial animation, allowing artists to skip tedious frame-by-frame work and focus on storytelling. Studios from toolmakers like Reallusion, as well as developers such as Survios and The Farm 51, say it speeds up pipelines, frees up animators for higher-value polish, and makes previously impractical ideas achievable, all while keeping characters feeling authentic and emotionally grounded. See how you can get started here. Check out the Audio2Face model in action below:


12 Comments on NVIDIA Wants Everyone to Own a Digital Avatar with Open-Source Audio2Face Animation Model
Also, a bit disappointing this is being "released" on Github, instead of the Nvidia app. AI tools like this won't gain widespread adoption until nontechnical users like myself can just download a single app and start using it, instead of having to go through GitHub or another technical setup process.
I had to struggle to setup Automatic1111, and ever since it broke a few months ago I haven't been able to fix it. Intel has an app called "AI Playground," which is a much better solution. You can download LLMs and image models in one place, and it automatically configures them for you.
It could process all 400 samples of "yes"/"no" in the dataset in one second (including loading/closing each file) with approx 50% exact pickup, 90% near, 1% error. No guessing whatsoever, so 50% is actually a good number.
<1ms processing time per sample allows the encoding of phonemes into the audio buffer in real time to be sent across a network.
Lipsync code can then play the audio file and phoneme animation at the same time,... and even play the lipsync animation first and delay the audio per the speed of sound without extra delays.
It can only do "yes/no" although all vowels are defined. I gave up after I couldn't find funding or a future path for it.
But this "more data, more power" approach from Nvidia, when games already max out a PC is not the future...
Its not quite uncanny valley, is it... The only difference is a higher resolution face, but underneath are very simple face physics and it all just identifies as a talking string puppet at best.