Tuesday, December 2nd 2025

AWS Releases Trainium3 ASIC to Ease Reliance on NVIDIA Hardware

During its re:Invent conference in Las Vegas, AWS introduced its latest ASIC chip, Trainium3, designed for internal AI workloads and select external customers. This chip delivers 2.52 PetaFLOPS of FP8 compute per chip and increases on-chip memory capacity to 144 GB of HBM3E, with a memory bandwidth of 4.9 TB/s. Trainium3 supports both dense and expert-parallel model topologies and introduces compact data types, MXFP8 and MXFP4, enhancing the balance between memory and compute for real-time, multimodal, and long-context reasoning tasks. The chip is manufactured using TSMC's N3 3 nm node and is now available in Amazon EC2 Trn3 UltraServer instances.

Trn3 UltraServers can scale up to 144 Trainium3 chips in a single server, achieving approximately 362 FP8 PetaFLOPS. These servers can be combined into EC2 UltraClusters 3.0 for larger deployments. A fully equipped UltraServer offers about 20.7 TB of HBM3e memory and around 706 TB/s of aggregate memory bandwidth. It also features the NeuronSwitch-v1 fabric, which doubles the interchip interconnect bandwidth compared to the previous UltraServer. AWS reports significant generational improvements, with up to 4.4x higher performance, 3.9x greater memory bandwidth, and about 4x better performance per watt compared to the Trainium2. Additionally, there are notable enhancements in inference and token efficiency for various Amazon services.
Amazon is providing these ASICs to help customers diversify their computing infrastructure beyond traditional NVIDIA and AMD GPUs. The company is already using this hardware to train and deploy its internal models and is also offering this computing power to some of the leading AI labs. Amazon has invested nearly $8 billion in Anthropic, which has been training and deploying some of its top models on AWS infrastructure, such as Trainium2. With the introduction of Trainium3, customers like Anthropic can expect significant efficiency improvements, which may encourage more major labs to adopt AWS infrastructure.
Source: AWS
Add your own comment

8 Comments on AWS Releases Trainium3 ASIC to Ease Reliance on NVIDIA Hardware

#1
TheGeekn°72
I'm curious about the cost of this design, it definitely can't be as priced as Rubin but I'm still curious what's the BOM on those things
Posted on Reply
#2
LastDudeALive
TheGeekn°72I'm curious about the cost of this design, it definitely can't be as priced as Rubin but I'm still curious what's the BOM on those things
Same node as Rubin, so BOM from TSMC will be the same, the chips are probably about the same size, looks like. On the plus side, you don't have to pay for Nvidia networking and CUDA, so the cost to the customer should be less than Rubin. On the downside, you don't get Nvidia networking and CUDA. Although Amazon absolutely has the resources to make their own in-house alternative, same as Google.
Posted on Reply
#3
TheGeekn°72
LastDudeALiveSame node as Rubin, so BOM from TSMC will be the same, the chips are probably about the same size, looks like. On the plus side, you don't have to pay for Nvidia networking and CUDA, so the cost to the customer should be less than Rubin. On the downside, you don't get Nvidia networking and CUDA. Although Amazon absolutely has the resources to make their own in-house alternative, same as Google.
Or probably get solarflare from AMD, they already get Epycs from them so...
Posted on Reply
#5
igormp
LastDudeALiveAlthough Amazon absolutely has the resources to make their own in-house alternative, same as Google.
TheGeekn°72Or probably get solarflare from AMD, they already get Epycs from them so...
For Trainium 2 instances on their "UltraServer" offerings, it had an interconnect called "NeuronLink" for the c2c interconnect, as well as their "elastic fabric" for networking. Both of these are proprietary and made in-house, as far as I'm aware.
Their GB200 "UltraServer" offering used NVLink for C2C and a beefed up version of their elastic fabric.

From the OP, I believe they'll be replacing that "NeuronLink" with the following:
It also features the NeuronSwitch-v1 fabric, which doubles the interchip interconnect bandwidth compared to the previous UltraServer.
In another announcment of theirs, they mention that for Trainium4 they'll be using NVLink:
To deliver even greater scale-up performance, Trainium4 is being designed to support NVIDIA NVLink Fusion high-speed chip interconnect technology. This integration will enable Trainium4, Graviton, and Elastic Fabric Adapter (EFA) to work together seamlessly within common MGX racks, providing you with a cost-effective, rack-scale AI infrastructure that supports both GPU and Trainium servers.
www.aboutamazon.com/news/aws/trainium-3-ultraserver-faster-ai-training-lower-cost
Posted on Reply
#6
lambda
igormpFor Trainium 2 instances on their "UltraServer" offerings, it had an interconnect called "NeuronLink" for the c2c interconnect, as well as their "elastic fabric" for networking. Both of these are proprietary and made in-house, as far as I'm aware.
Their GB200 "UltraServer" offering used NVLink for C2C and a beefed up version of their elastic fabric.

From the OP, I believe they'll be replacing that "NeuronLink" with the following:


In another announcment of theirs, they mention that for Trainium4 they'll be using NVLink:

www.aboutamazon.com/news/aws/trainium-3-ultraserver-faster-ai-training-lower-cost
- Afaik the internal fabric has some issues.
- Unsure about the larger customers, Anthropic uses a good number of these chips but most large customers still prefer Nv due to the software stack.
Posted on Reply
#7
igormp
lambda- Afaik the internal fabric has some issues.
- Unsure about the larger customers, Anthropic uses a good number of these chips but most large customers still prefer Nv due to the software stack.
I can't say anything about your first point since I've never tried using any of their UltraServers for comms to matter.
As for the second, my experiences with Trainium instances were annoying and I don't personally know anyone using those. I assume most of those are being used internally for their bedrock/sagemaker managed solutions.
Posted on Reply
#8
lambda
igormpI can't say anything about your first point since I've never tried using any of their UltraServers for comms to matter.
As for the second, my experiences with Trainium instances were annoying and I don't personally know anyone using those. I assume most of those are being used internally for their bedrock/sagemaker managed solutions.
First one is for internal scaling runs or things I've heard through the grapevine. Maybe that's why they moved to NVlink.

Your second point is correct afaik, I don't know any big external customers using it. Most of the ones I interact with prefer NV.
Posted on Reply
Nov 20th, 2026 23:08 PST change timezone

New Forum Posts

Popular Reviews

TPU on YouTube

Controversial News Posts