Wednesday, September 24th 2025

NVIDIA Wants Everyone to Own a Digital Avatar with Open-Source Audio2Face Animation Model

NVIDIA is today open-sourcing its Audio2Face 3D animation model, allowing everyone to become the owner of their own digital avatar. The company has trained its Audio2Face technology to produce characters that are true to life, utilizing generative video models for facial expressions, text-to-speech models for human output, and large language models for actual conversations, all combined in a single pipeline to create a digital avatar. All of these stages are fed by analyzing acoustic input, which features characteristics such as intonation and phonemes, and are then translated into a stream of animation data. That data is later mapped to facial expressions, where the mood of the avatar can be interpreted simply by looking at it. The entire data pipeline can be either an offline scripted content or you can feed the model using live data for real-time lip-synching and facial expressions.

Additionally, NVIDIA is open-sourcing the entire Audio2Face stack, which includes the model itself and the SDK, so everyone, from game developers to home enthusiasts, can use it in any possible way. NVIDIA is now open-sourcing the entire training framework, allowing anyone to create their own avatar version with customized features. The company even provided Autodesk Maya and Unreal Engine plugins, allowing 3D workloads to integrate this generative model immediately.
Game developers are increasingly treating Audio2Face as a production-ready shortcut to believable facial performance—a tool that automatically converts dialogue into expressive, multilingual lip-sync and facial animation, allowing artists to skip tedious frame-by-frame work and focus on storytelling. Studios from toolmakers like Reallusion, as well as developers such as Survios and The Farm 51, say it speeds up pipelines, frees up animators for higher-value polish, and makes previously impractical ideas achievable, all while keeping characters feeling authentic and emotionally grounded. See how you can get started here. Check out the Audio2Face model in action below:
Add your own comment

12 Comments on NVIDIA Wants Everyone to Own a Digital Avatar with Open-Source Audio2Face Animation Model

#1
LastDudeALive
The company has trained its Audio2Face technology to produce characters that are true to life
Meanwhile, the video shows a face that looks like it has more Botox than an 80 year old Hollywood actress.

Also, a bit disappointing this is being "released" on Github, instead of the Nvidia app. AI tools like this won't gain widespread adoption until nontechnical users like myself can just download a single app and start using it, instead of having to go through GitHub or another technical setup process.

I had to struggle to setup Automatic1111, and ever since it broke a few months ago I haven't been able to fix it. Intel has an app called "AI Playground," which is a much better solution. You can download LLMs and image models in one place, and it automatically configures them for you.
Posted on Reply
#2
dgianstefani
TPU Proofreader
LastDudeALiveMeanwhile, the video shows a face that looks like it has more Botox than an 80 year old Hollywood actress.

Also, a bit disappointing this is being "released" on Github, instead of the Nvidia app. AI tools like this won't gain widespread adoption until nontechnical users like myself can just download a single app and start using it, instead of having to go through GitHub or another technical setup process.

I had to struggle to setup Automatic1111, and ever since it broke a few months ago I haven't been able to fix it. Intel has an app called "AI Playground," which is a much better solution. You can download LLMs and image models in one place, and it automatically configures them for you.
Personally I would prefer the driver app to stay clean and light. Not bloated with every single possible tool usable with the GPU. Nothing wrong with plugins or github, these tools are geared towards developers anyway.
Posted on Reply
#3
Vayra86
Euh, nope I'll pass Huang.
Posted on Reply
#4
RGAFL
LastDudeALiveMeanwhile, the video shows a face that looks like it has more Botox than an 80 year old Hollywood actress.

Also, a bit disappointing this is being "released" on Github, instead of the Nvidia app. AI tools like this won't gain widespread adoption until nontechnical users like myself can just download a single app and start using it, instead of having to go through GitHub or another technical setup process.

I had to struggle to setup Automatic1111, and ever since it broke a few months ago I haven't been able to fix it. Intel has an app called "AI Playground," which is a much better solution. You can download LLMs and image models in one place, and it automatically configures them for you.
If you want a frindlier way of playing with AI image generation use Stability Matrix or Pinokio. All the Python dependencies etc are handled by them. Pretty foolproof way of getting into AI image generation. Also Automatic1111 although still widely used is not currently updated by it's author. I personally use SwarmUI or Comfy.
Posted on Reply
#6
tpa-pr
RGAFLIf you want a frindlier way of playing with AI image generation use Stability Matrix or Pinokio. All the Python dependencies etc are handled by them. Pretty foolproof way of getting into AI image generation. Also Automatic1111 although still widely used is not currently updated by it's author. I personally use SwarmUI or Comfy.
I also suggest EasyDiffusion, I stumbled across it when my StabilityMatrix install broke and couldn't be repaired. It was very one-click-and-go setup.
Posted on Reply
#7
mikesg
1-2 years ago I developed basic ASR + lipsync without AI. Focusing on "yes/no" using Googles Speech Commands dataset.

It could process all 400 samples of "yes"/"no" in the dataset in one second (including loading/closing each file) with approx 50% exact pickup, 90% near, 1% error. No guessing whatsoever, so 50% is actually a good number.

<1ms processing time per sample allows the encoding of phonemes into the audio buffer in real time to be sent across a network.

Lipsync code can then play the audio file and phoneme animation at the same time,... and even play the lipsync animation first and delay the audio per the speed of sound without extra delays.

It can only do "yes/no" although all vowels are defined. I gave up after I couldn't find funding or a future path for it.

But this "more data, more power" approach from Nvidia, when games already max out a PC is not the future...
Posted on Reply
#8
Vayra86
Strangely when I see those faces I can't help but think of Fallout or Mass Effect Andromeda.

Its not quite uncanny valley, is it... The only difference is a higher resolution face, but underneath are very simple face physics and it all just identifies as a talking string puppet at best.
Posted on Reply
#9
RGAFL
tpa-prI also suggest EasyDiffusion, I stumbled across it when my StabilityMatrix install broke and couldn't be repaired. It was very one-click-and-go setup.
Easy Diffusion is not very friendly with Flux and Qwen Image support is non existent. SwarmUI and Comfy works well with both. Flux and Qwen Image are light years ahead of Stable Diffusion for image generation.
Posted on Reply
#10
LastDudeALive
RGAFLIf you want a frindlier way of playing with AI image generation use Stability Matrix or Pinokio. All the Python dependencies etc are handled by them. Pretty foolproof way of getting into AI image generation. Also Automatic1111 although still widely used is not currently updated by it's author. I personally use SwarmUI or Comfy.
tpa-prI also suggest EasyDiffusion, I stumbled across it when my StabilityMatrix install broke and couldn't be repaired. It was very one-click-and-go setup.
Thanks for the suggestions! I installed Stability Matrix last night, and seems to work good so far! Having the entire installation in a single folder that can be moved was very nice.
Posted on Reply
#12
RGAFL
LastDudeALiveThanks for the suggestions! I installed Stability Matrix last night, and seems to work good so far! Having the entire installation in a single folder that can be moved was very nice.
I'd also suggest if doing human image generation and you want to use the same person then use Flux and train a Lora. That way you get to keep the likeness across generations. I'll post some photos as an example.
Posted on Reply
Add your own comment
Nov 20th, 2026 14:59 PST change timezone

New Forum Posts

Popular Reviews

TPU on YouTube

Controversial News Posts