News Posts matching #Instinct

Return to Keyword Browsing

AMD Details Single Node & Distributed Inference Performance on Instinct MI355X GPU

The rapid growth of GenAI and LLM workloads, from agentic workflows and multi-step tool use to retrieval-augmented reasoning, demands inference infrastructure that is fast, adaptable, and highly optimized. While AMD continues to invest in general-purpose inference frameworks such as vLLM and SGLang, ATOM provides the most direct path to peak Instinct MI355X GPU performance for modern reasoning and MoE-heavy workloads, which increasingly dominate frontier LLM architectures.

Over the past months, AMD have implemented numerous optimizations to improve both single-node performance and multi-node distributed inference for DeepSeek-R1 on the MI355X GPU.

NVIDIA Powers Nearly 72% of Cloud Accelerator Locations, AMD at 5.8%

The cloud accelerator market share study was conducted at UBS bank, one of the leading research hubs and close followers of market trends. According to UBS, NVIDIA commands 71.2% of the cloud accelerator location share, which includes GPUs and other types of ASICs. The study also included AMD, and other ASICs. For the AMD front, the situation is entirely different as the data shows AMD only captures 5.8% of the cloud accelerator location share. The remaining portion is split between ASICs designed by the likes of AWS, Meta, Broadcom, MediaTek, and many others, which command 22.3%.

For NVIDIA, the company has managed to penetrate its GPUs in 258 datacenter locations, serving as cloud GPU providers. This is a significant number considering that some of these customers are ordering tens and even hundreds of thousands of GPUs at once, especially as the demand expands. NVIDIA is still the backbone of the entire AI infrastructure, and even its older GPUs, which include A100 and H100, are capturing a significant part of the GPU cloud market share. As "Blackwell" SKUs, designed for a million GPU scale systems, are scaled, and we enter the "Rubin" era, NVIDIA's lead may only expand.

NVIDIA Reaffirms Support for FP64, Next-Gen GPU to Bring HPC Improvements

NVIDIA's recent generations of GPUs, such as "Hopper" and "Blackwell," have been known for stalling their high-performance computing (HPC) oriented FP64 double-precision performance. However, the company is not inherently abandoning the HPC space and has confirmed for HPCWire that 64-bit floating point data is not going away. According to Dion Harris, the senior director of HPC and AI hyperscale infrastructure solutions at NVIDIA, the company is "definitely looking to bring some additional [FP64] capabilities in our future gen architectures. We are very serious about making sure that we can deliver the required performance to power those simulation workloads."

The acceleration of 64-bit floating-point data paths is crucial for the HPC community, particularly in the life sciences. When a workload demands sustained high-precision support, NVIDIA's recent generations have not met expectations. For comparison, NVIDIA's current most powerful B300 "Blackwell Ultra" accelerator achieves only 1.2 TeraFLOPS of FP64 performance. In contrast, the older H200 "Hopper" reaches an impressive 34 TeraFLOPS of FP64 compute at its peak. For FP8 low-precision, the B300 delivers 9 PetaFLOPS, while the H200 provides 3.958 PetaFLOPS.

AMD Instinct MI355X Drives Competitive Performance, Ideal for Optimized LLM Workloads

AI training workloads are pushing the limits of modern GPU architectures. With the release of AMD ROCm 7.0 software, AMD is raising the bar for high-performance training by delivering optimized support for LLM workloads across the JAX and PyTorch frameworks. The latest v25.9 Training Dockers demonstrate exceptional scaling efficiency for both single-node and multi-node setups, empowering researchers and developers to push model sizes and complexity further than ever.

By integrating Primus, a unified and flexible LLM training framework, the PyTorch training docker streamlines LLM development on AMD Instinct GPUs. Primus now supports both TorchTitan and Megatron-LM backends, while Primus-Turbo accelerates Transformer models, further boosting training throughput on AMD Instinct MI355X GPUs. Try Primus-Repo here.

HLRS Announces Details of Herder Supercomputer

The High-Performance Computing Center Stuttgart (HLRS) today announced the architecture of HLRS's upcoming next-generation supercomputer, developed with its technology partners, HPE and AMD. The new system—called Herder—will offer both a major increase in performance and an optimized architecture for today's most advanced computational applications, giving German and European research and industry a powerful new tool for scientific discovery and industrial innovation.

Herder will be based on the HPE Cray Supercomputing GX5000 system. Announced by HPE in November 2025, it is designed to support the large-scale numerical simulations that are essential within HLRS's traditional user communities, as well as data-science approaches for AI model training and generative AI. Bringing these capabilities together within a single system enables the development of new kinds of converged computing workflows that seamlessly integrate the two approaches.

Next-Gen HBM Targets Embedded GPU Cores

Tech companies are exploring a major shift in HBM design seeking to embed GPU cores directly into next-generation memory stacks. According to Korean industry reports, Meta and NVIDIA are evaluating "custom HBM" architectures that integrate GPU cores into the base die of future HBM devices, with SK Hynix and Samsung said to be involved in early discussions. HBM stacks multiple DRAM dies on top of a base die that handles external I/O. HBM4 is expected to reach mass production next year and will include an onboard controller to improve bandwidth and efficiency. Integrating GPU cores would take this concept several steps further, distributing compute into the memory itself to reduce data movement and cut power usage. Industry sources say the approach could boost both performance and energy efficiency for AI workloads by cutting the distance between compute and memory. But the design still faces major struggles such as limited die area in Through-Silicon Vias (TSV) based stacks, power delivery constraints, and the difficulty of cooling those compute-heavy GPU logic inside the base die.

Kim Joung-ho, a professor in the School of Electrical Engineering at KAIST, said, "The speed of technological transition where the boundary between memory and system semiconductors collapses for AI advancement will accelerate," and added, "Domestic companies must expand their ecosystem beyond memory into the logic sector to preempt the next-generation HBM market." AMD's recently unveiled Instinct MI430X accelerator built on the next-generation AMD CDNA architecture supports 432 GB of HBM4 memory and 19.6 TB/s of memory bandwidth. NVIDIA's "Vera Rubin" Superchip takes a different approach. Each Rubin GPU integrates two reticle-sized compute chiplets paired with eight HBM4 stacks, delivering around 288 GB of HBM4 per GPU and roughly 576 GB of HBM4 across the full Superchip. From those potential changes companies that have strong packaging and logic capabilities will benefit, while pure memory vendors may need to expand into system-level semiconductor technologies to stay competitive.

TOP500: El Capitan Retains #1 as JUPITER Becomes Europe's First Exascale System

The 66th edition of the TOP500 list of the world's most powerful supercomputers was announced today at the SC25 Conference in St Louis, Missouri. The new list reflects continued U.S. leadership in high-performance computing (HPC), historic European milestones, and growing global diversity across architectures and energy-efficient design.

El Capitan Extends Leadership Across Benchmarks
The El Capitan system at Lawrence Livermore National Laboratory (LLNL) remains the undisputed leader of global HPC. Based on the HPE Cray EX255a architecture, powered by AMD 4th Gen EPYC CPUs and AMD Instinct MI300A accelerators, El Capitan achieved 1.809 Exaflop/s on the HPL benchmark—a significant remeasurement that further strengthens its position as the fastest computer in the world. El Capitan also dominates the HPCG benchmark, delivering 17.41 HPCG-Petaflop/s, making it the #1 for real-world application performance. On the HPL-MxP mixed-precision benchmark, it recorded 16.7 Exaflop/s, confirming its versatility in AI and data-driven workloads. With 11.34 million cores and an energy efficiency of 60.9 GFlops/Watt, El Capitan exemplifies the current generation of exascale computing.

AMD and Eviden to Power Europe's New Exascale Supercomputer

AMD (NASDAQ: AMD) and Eviden, the Atos Group product brand leading in advanced computing, announced their selection to build Alice Recoque, a next-generation supercomputer to support the need for high performance computing (HPC) and AI, serving as an AI Factory. Alice Recoque will be France's first and Europe's second Exascale supercomputer, led by Grand équipement national de calcul intensif (GENCI), operated by Commissariat à l'énergie atomique et aux énergies alternatives (CEA), and powered by next-generation AMD AI and HPC compute technologies.

This project, representing an overall cost of 544 million euros, is funded by EuroHPC JU, with budget stemming from the Digital Europe Programme (DEP), and the Jules Verne Consortium, led by France through GENCI and CEA with the participation of Netherlands IT cooperation SURF and Greece with GRNET.

AMD Unveils First MLPerf 5.1 Training Results on Instinct MI350 Series GPUs

The latest MLPerf 5.1 Training results mark an important milestone—the first MLPerf training submission using AMD Instinct MI350 Series GPUs. These new benchmarks demonstrate breakthrough generational performance gains and highlight broad ecosystem participation across some of today's most demanding AI training workloads.

Building on the strong MLPerf 5.1 Inference results released earlier this year, this submission represents the first time the AMD Instinct MI350 Series—including both the MI355X and MI350X GPUs—have been publicly benchmarked for AI training. The results show clear progress in scalability, efficiency, and compute performance, underscoring how the AMD Instinct MI350 Series is accelerating the development of next-generation generative AI models.

HPE Unveils Newest Additions to Its Next-Generation Cray Supercomputing Portfolio

HPE (NYSE:HPE) today announces the newest additions to the next-generation HPE Cray supercomputing portfolio that provide industry-leading compute density, designed to meet the needs of artificial intelligence (AI) demands while performing at-scale. This HPE Cray Supercomputing portfolio expansion includes three multipartner, multi-workload compute blades, unified management software and high-performance interconnect support, establishing one of the most powerful supercomputing architectures in the industry for research laboratories, sovereign entities and large enterprises that are increasingly turning to AI to advance their simulations and drive scientific discovery.

"Global organizations relying on supercomputing are looking for better computing performance for all of their workloads," said Trish Damkroger, senior vice president and general manager, HPC and AI Infrastructure Solutions at HPE. "Our new HPE Cray Supercomputing platform is the answer to customers' needs for higher performance density with a unified AI and HPC architecture that is engineered for groundbreaking outcomes. HPE is proud to share our unwavering commitment to AI and supercomputing as the engine of innovation and scientific discovery, advancing the way people live and work."

AMD Claims AI Fuels Soaring Growth in $60 Billion Server CPU Market

The global data center expansion is not only accelerating GPU deployments, but also CPUs. At AMD's financial analyst day yesterday, the company presented an interesting chart that shows massive growth rate for the regular CPUs amid the AI boom. As a universal base platform for every system, CPUs have been somewhat overlooked, with GPUs taking much of the attention. AMD's internal projection shows that AI inflection will drive an additional $30 billion of revenue for CPU demand exclusively, by 2030. This number is supposed to stack on the already existing regular server CPU demand—sitting at about $30 billion—making the 2030 projections stand at a staggering $60 billion of revenue by 2030, for CPUs alone.

The company's analysis shows that the $19 billion CPU sector from 2022 is rapidly evolving, with AI-driven CPU workloads fueling an 18% annual growth rate that will shape this decade. This is more than just incremental expansion. It shows a shift in enterprise computing priorities and a continued rise in demand for traditional hardware. While the conventional general compute segment is growing at a modest 4% CAGR, the AI-driven portion is advancing rapidly, reaching an inflection point that will allow it to capture the majority of data center CPU value by the end of the decade.

AMD Positions Instinct MI400 Against NVIDIA Vera Rubin, MI500 Coming in 2027

At its Financial Analyst Day 2025, AMD shared new details about its upcoming Instinct MI400 and MI500 AI accelerators. The Instinct MI400 series set to launch in 2026, will be based on the new CDNA 5 architecture. AMD claims it will deliver up to 40 FP4 and 20 FP8 PFLOPs, roughly twice the compute performance of the current MI350. The GPUs also move to HBM4 memory from HMB3e, increasing capacity from 288 GB to 432 GB and raising total bandwidth from 8 TB/s to 19.6 TB/s. Each GPU provides 300 GB/s of scale-out bandwidth and adds broader AI data format support along with expanded AI pipelines. Two models are planned: the Instinct MI455X for large-scale AI training and inference, and the MI430X for HPC and sovereign AI. The MI430X features FP64 hardware based support and hybrid CPU+GPU compute capability, while retaining the same HBM4 memory subsystem as the MI455X. The MI400 series also moves away from the CoWoS-S to the CoWoS-L (Local Silicon Interconnect) advanced packaging technology.

AMD compares the MI400 lineup to NVIDIA's upcoming Vera Rubin series, claiming similar compute performance and memory bandwidth, but 1.5× higher memory capacity and scale-out bandwidth. During her presentation, Dr. Lisa Su said that AMD's Instinct MI400 accelerators will be available to clients and partners from day one, while the next-generation MI500 is already in advanced design stages. AMD is also now moving to an annual refresh cycle for its Instinct products, confirming that its next-gen MI500 series will be launched in 2027.

AMD Secures $1B U.S. Energy Department Contract for Lux and Discovery Supercomputers

AMD's expertise in the supercomputers sector seems to pay off as the company secured a $1 billion partnership with the U.S. Department of Energy (DoE) to build two next-generation supercomputers aimed at scientific and national security challenges, from nuclear fusion research to cancer treatments. Energy Secretary Chris Wright said the initiative will "supercharge" progress in nuclear energy, defense technologies and drug discovery, accelerating breakthroughs that traditionally take decades. The first system, Lux, is set to go online within six months. Co-developed by AMD, Hewlett Packard Enterprise, Oracle Cloud Infrastructure, and Oak Ridge National Laboratory (ORNL), Lux will be powered by AMD's Instinct MI355X AI accelerators alongside AMD CPUs and networking chips. ORNL Director Stephen Streiffer noted that Lux will deliver roughly three times the AI performance of today's top supercomputers, while AMD CEO Lisa Su called it "the fastest deployment of this scale we've ever seen." A second system, dubbed Discovery, will follow later in the decade. Built on AMD's upcoming MI430 AI chips, the more advanced machine is expected to be completed by 2028 and operational by 2029. Both systems are part of a broader U.S. effort to maintain leadership in high-performance computing and artificial intelligence.

According to Reuters, the DoE's choice of AMD reflects confidence in the company's proven HPC track record, particularly with Frontier and El Capitan, the world's fastest supercomputers, which also runs on AMD hardware. Wright added that these new systems will help simulate complex processes like plasma dynamics for fusion energy and molecular interactions for drug development. "We're going to see massively faster progress toward harnessing fusion and treating diseases that are still death sentences today," he said. Together, Lux and Discovery mark a milestone for both AMD and U.S. scientific computing and build momentum for AMD's upcoming MI400 accelerators series.

AMD Showcases "Helios" Rack-Scale Platform

Today at the Open Compute Project (OCP) Global Summit in San Jose, AMD (NASDAQ: AMD) showcased a static display of its "Helios," rack scale platform for the first time in public. Developed based on the new Open Rack Wide (ORW) specification, introduced by Meta, "Helios" extends the AMD open hardware philosophy from silicon to system to rack, representing a major step forward in open, interoperable AI infrastructure.

Extending AMD leadership in AI and high-performance computing, "Helios" provides the foundation to deliver the open, scalable infrastructure that will power the world's growing AI demands. Designed to meet the demands of gigawatt-scale data centers, the new ORW specification defines an open, double-wide rack optimized for the power, cooling, and serviceability needs of next-generation AI systems. By adopting ORW and OCP standards, "Helios" provides the industry with a unified, standards-based foundation to develop and deploy efficient, high-performance AI infrastructure at scale.

Oracle to Deploy 50,000 AMD Instinct MI450 GPUs in New AI Superclusters

AMD and Oracle have expanded their ongoing partnership aiming to grow AI infrastructure through big GPU clusters on Oracle Cloud Infrastructure (OCI). OCI will host the first AI supercluster available to the public that runs on AMD Instinct MI450 Series GPUs. The plan is to start with 50,000 units in Q3 2026 and expand further in 2027. The new systems will build on Oracle's existing AMD-based offerings, including Instinct MI300X and MI355X GPU instances, and introduce AMD's next-generation "Helios" rack design. Each rack combines Instinct MI450 GPUs, EPYC "Venice" CPUs, and Pensando "Vulcano" DPUs using liquid-cooled setups to boost performance while reducing costs.

AMD's upcoming MI450 accelerators series boasts up to 432 GB of HBM4 memory and 20 TB/s of bandwidth for each GPU. This allows for the training of much bigger AI models in-memory while AMD's UALink and UALoE interconnect standards make it possible for GPUs to communicate with hardware coherence. This cuts down on latency and CPU routing. Each GPU can also team up with up to three 800 Gbps Pensando "Vulcano" AI-NICs. These support advanced RoCE and UEC networking standards to enable high-throughput distributed training. AMD EPYC "Venice" CPUs with features for confidential computing and AMD's ROCm open-source software stack will complement the new units.

OpenAI will Buy AMD GPUs, AMD Offers $30+ Billion in Stock to Sweeten the Deal

AMD (NASDAQ: AMD) and OpenAI today announced a 6 gigawatt agreement to power OpenAI's next-generation AI infrastructure across multiple generations of AMD Instinct GPUs. The first 1 gigawatt deployment of AMD Instinct MI450 GPUs is set to begin in the second half of 2026. AMD's strong leadership in high-performance computing systems and OpenAI's pioneering research and advancements in generative AI places the two companies at the forefront of this important and pivotal time for AI.

Under this definitive agreement, OpenAI will work with AMD as a core strategic compute partner to drive large-scale deployments of AMD technology starting with the AMD Instinct MI450 series and rack-scale AI solutions and extending to future generations. By sharing technical expertise to optimize their product roadmaps, AMD and OpenAI are deepening their multi-generational hardware and software collaboration that began with the MI300X and continued with the MI350X series. This partnership creates a true win-win for both companies, enabling very large-scale AI deployments and advancing the entire ecosystem.

AMD Enables PyTorch on Radeon RX 7000/9000 GPUs with Windows and Linux Preview

AMD has rolled out a public preview of ROCm 6.4.4 that finally lets PyTorch run natively on Windows and Linux for a broad swath of its consumer silicon. The update brings official framework support to Radeon RX 9000 (RDNA 4) and RX 7000 (RDNA 3) GPUs, as well as to select Ryzen AI 300 "Strix" and Ryzen AI MAX "Strix Halo" APUs, fulfilling a promise AMD made at Computex 2025 to make ROCm more cross-platform and developer-friendly. "At Computex this year, I shared our commitment to making ROCm a true cross-platform, developer-first stack. I said we'd bring ROCm to Radeon on Windows and Linux in the second half of 2025. I'm proud to say that today, we're delivering on that promise," says Andrej Zdravkovic, Senior Vice President and Chief Software Officer.

For Windows users, this is a genuine quality-of-life change: no more dual-booting, convoluted workarounds, or constant reliance on cloud instances to test and run AI models locally. AMD describes the release as a foundation rather than a finished product. The preview provides native PyTorch wheels, allowing developers to prototype, benchmark, and provide feedback while AMD iterates on performance and feature coverage. The company positions this step as part of a broader effort to extend ROCm beyond the data center and into the everyday machines of creators and developers. At the same time, enterprise customers continue to get higher-scale optimizations through the ROCm 7.0 family for Instinct and EPYC platforms. If community testing is successful, users can expect more frequent updates and enhanced Windows support in the next release cycle.

AMD and Cohere Expand Global AI Collaboration to Power Enterprise and Sovereign Deployments with AMD AI Infrastructure

Today, AMD announced it is expanding its global collaboration on multiple fronts with Cohere, the leading security-first enterprise AI company. Cohere customers will have access to AMD Instinct GPU-powered infrastructure for Cohere's full suite of AI offerings, including Command A, Command A Vision, Command A Translate and North. This gives enterprise and sovereign AI initiatives in Canada and around the world more choices to meet performance and TCO goals that make their AI plans real.

AMD also announced that it will integrate Cohere's North platform into its internal enterprise AI portfolio, making Cohere's technology an integral part of the AMD internal and engineering AI workloads at AMD.

AMD Launches ROCm 7.0, Up to 3.8x Performance Uplift Over ROCm 6.0

AMD today unveiled ROCm 7.0, a massive update to its open GPU software platform for AI workloads across datacenter racks and even client devices. The release focuses on support for the new Instinct MI350 accelerator series, built on CDNA 4. It pairs hardware-level refinements, including advanced HBM utilization and expanded datatype handling, with software improvements designed to enhance throughput for both training and inference. AMD classifies this release as a generational step that widens deployment choices for enterprises, from single-node servers to multi-rack clusters. Company benchmarks shared with partners indicate multi-fold gains over previous ROCm 6.0 versions on select large-model tasks, and AMD shows tighter integration with open-source serving stacks to help teams transition from prototype to production more quickly.

Under the hood, ROCm 7.0 delivers algorithmic and systems advances intended to reduce cost and improve utilization: enhanced attention and reasoning primitives, sparse mixture-of-experts techniques, and support for FP4, FP6, and FP8 precision formats that trade a bit of accuracy for massive memory savings. ROCm 7.0 also focuses on scale and manageability by enabling GPU-direct communications, distributed inference across nodes with vLLM-d and DeepEP compatibility, and cluster tools such as AMD Resource Manager and a new AI Workbench for orchestration and monitoring. To smooth adoption, AMD provides prebuilt Docker images with quantized model examples and turnkey stacks for common frameworks. AMD is also expanding endpoint support to Ryzen AI processors and Radeon RX series GPUs, so developers can deploy consistent pipelines from edge devices to cloud racks. ROCm 7 is available now through AMD's developer resources, with documentation, examples, and enterprise support to help teams get started.

AMD Instinct MI500 UAL256 Mega Pod to Scale up to 256 GPUs, 64 "Verano" CPUs

While AMD hasn't launched its upcoming Instinct MI400 series of accelerators yet, the company has started preparing its supply chain for what comes after. Thanks to SemiAnalysis, we have a clearer picture of what AMD's Instinct MI500 UAL256 rack will look like. At the base of each compute node, there will be a single "Verano" CPU paired with four Instinct MI500 accelerators. With up to 64 nodes, split across two 32-node racks, AMD plans to enable 256 logical GPUs and 64 CPUs per mega pod, working in tandem for advanced AI and HPC workloads. Instinct MI500s UAL256 will be AMD's second-generation rack-scale system, following the upcoming Instinct MI450X IF128 aka "Helios", scheduled for the second half of 2026. The Instinct MI500 UAL256 scale-up mega pod is scheduled to arrive in 2027, just in time for Verano and Instinct MI500 accelerator.

The Instinct MI500 UAL256 scale-up mega pod splits the compute into two 32-rack cabinets, with a single central rack connecting them and hosting 18 switching trays for networking. Each of these networking trays contains four 102.4T "Vulcano" switch ASICs, designed for 800G external throughput, and manufactured on TSMC's 3 nm node. For the central processor, AMD showed few details at its Advancing AI event, which claimed that Verano introduces the "Zen 7" microarchitecture for even higher IPC and support for even newer instruction sets. For now, it's not clear if AMD will increase CPU core counts beyond the up to 256 cores per package of "Venice." Still, we're hearing that Verano will retain the Socket SP7 infrastructure, which means it will likely retain the memory and PCIe interfaces introduced in 2026, such as PCIe Gen 6, UALink, and Ultra Ethernet.

AMD's Next-Gen UDNA: Four Die Sizes, One Potential 96-CU Flagship

New block diagrams from a well-known hardware leaker @Kepler_L2 sketch out a very modular AMD UDNA family that could include four die sizes, with the largest rumored to reach 96 compute units. The images show repeating shader arrays and shader engines, each engine holding a handful of compute units and its own render backend. Those engines feed into a central SoC block that contains the graphics command processor, the graphics engine, hardware schedulers, and a shared L2 cache. On the memory side, the flagship diagram shows sixteen unified memory controllers, each with 32 bits, which would add up to a 512-bit external interface. The leak also mentions the possibility of a much larger on-die Infinity Cache for the top part.

If you map the parts together, the flagship's numbers line up with the diagram. Eight shader arrays with two shader engines each give 16 shader engines, and with six compute units per engine, that reaches the 96 CU figure. The mid-tier design pares things back: four shader arrays and eight shader engines, with five compute units per engine, which is about 40 compute units and an estimated six memory controllers for a 192-bit bus. Below that are 24 CU and 12 CU designs built from smaller arrays and fewer controllers. The 24 CU design is shown with as many as eight memory controllers, which could mean different controller widths depending on whether the part uses a traditional GDDR style interface or an LPDDR5X-like arrangement.

NVIDIA and AMD Customize HBM4 Memory for Next-Generation AI Accelerators

NVIDIA's and AMD's next-generation accelerators will definitely utilize a next-generation HBM4 memory, too. However, interesting news, confirmed by SemiAnalysis, is that these accelerators will utilize a customized HBM memory for the first time. In an HBM stack, consisting of stacked DRAM dies up to a 12-high, connected with TSVs, there is the possibility of embedding an optional base die with customized logic/accelerator circuitry tailored to a specific need. It appears that NVIDIA and AMD are not only exploring this option to gain more performance, but are also actively working on an implementation with a custom base that will push these accelerators ahead of any third-party ASIC solution by a third. Since everyone is trying to grab a bit of market share in the AI ASIC world, NVIDIA and AMD have to innovate their designs to stay ahead constantly.

"We expect NVIDIA and AMD to come out with their custom HBM implementations with HBM4, with other accelerator designers likely to only have a custom base die implementation by HBM4E in the 2027 timeframe," notes the SemiAnalysis post. This means that while ASIC designers, such as Broadcom and MediaTek, will have a working solution in 2027 chips, NVIDIA and AMD will have a working custom HBM4 in 2026 products. For NVIDIA, this is the "Rubin" architecture, supposed to outperform the current "Blackwell" implementation by a wide margin. For AMD, this is the Instinct MI400 series of accelerators, which is coming for the first time in a rack-scale flavor.

Next-Gen HBM4 to HBM8: Toward Multi-Terabyte Memory on 15,000 W Accelerators

In a joint briefing this week, KAIST's Memory Systems Laboratory and TERA's Interconnection and Packaging group presented a forward-looking roadmap for High Bandwidth Memory (HBM) standards and the accelerator platforms that will employ them. Shared via Wccftech and VideoCardz, the outline covers five successive generations, from HBM4 to HBM8, each promising substantial gains in capacity, bandwidth, and packaging sophistication. First up is HBM4, targeted for a 2026 rollout in AI GPUs and data center accelerators. It will deliver approximately 2 TB/s per stack at an 8 Gbps pin rate over a 2,048-bit interface. Die stacks will reach 12 to 16 layers, yielding 36-48 GB per package with a 75 W power envelope. NVIDIA's upcoming Rubin series and AMD's Instinct MI500 cards are slated to employ HBM4, with Rubin Ultra doubling the number of memory stacks from eight to sixteen and AMD targeting up to 432 GB per device.

Looking to 2029, HBM5 maintains an 8 Gbps speed but doubles the I/O lanes to 4,096 bits, boosting throughput to 4 TB/s per stack. Power rises to 100 W and capacity scales to 80 GB using 16-high stacks of 40 Gb dies. NVIDIA's tentative Feynman accelerator is expected to be the first HBM5 adopter, packing 400-500 GB of memory into a multi-die package and drawing more than 4,400 W of total power. By 2032, HBM6 will double pin speeds to 16 Gbps and increase bandwidth to 8 TB/s over 4,096 lanes. Stack heights can grow to 20 layers, supporting up to 120 GB per stack at 120 W. Immersion cooling and bumpless copper-copper bonding will become the norm. The roadmap then predicts HBM7 in 2035, which includes 24 Gbps speeds, 8,192-bit interfaces, 24 TB/s throughput, and up to 192 GB per stack at 160 W. NVIDIA is preparing a 15,360 W accelerator to accommodate this monstrous memory.

AMD's Answer to AI Advancement: ROCm 7.0 Is Here

In August, AMD will release ROCm 7, its open computing platform for high-performance computing, machine learning, and scientific applications. This version will support a range of hardware, from Ryzen AI-equipped laptops to Radeon AI Pro desktop cards and server-grade Instinct GPUs, which have just received an update. Before the end of 2025, ROCm 7 will be integrated directly into Linux and Windows, allowing for a seamless installation process with just a few clicks. AMD isn't planning to update ROCm once every few months, either. Instead, developers will receive day-zero fixes and a major update every two weeks, complete with performance enhancements and new features. Additionally, a dedicated Dev Cloud will provide everyone with instant access to the latest AMD hardware for testing and experimentation.

Early benchmarks are encouraging. On one test, an Instinct MI300X running ROCm 7 reached roughly three times the speed recorded with the original ROCm 6 release. Of course, your mileage will vary depending on model choice, quantization, and other factors. This shift follows comments from AMD's Senior Vice President and Chief Software Officer, Andrej Zdravkovic, whom we interviewed last September. He emphasized ROCm's open-source design and the utility of HIPIFY, a tool that converts CUDA code to run on ROCm. This will enable a full-scale ROCm transition, now accelerated by a 3x performance uplift simply by updating the software version. If ROCm 7 lives up to its promise, AMD could finally unlock the potential of its hardware across devices, both big and small, and provide NVIDIA with good competition in the coming years.

Micron HBM Designed into Leading AMD AI Platform

Micron Technology, Inc. today announced the integration of its HBM3E 36 GB 12-high offering into the upcoming AMD Instinct MI350 Series solutions. This collaboration highlights the critical role of power efficiency and performance in training large AI models, delivering high-throughput inference and handling complex HPC workloads such as data processing and computational modeling. Furthermore, it represents another significant milestone in HBM industry leadership for Micron, showcasing its robust execution and the value of its strong customer relationships.

Micron HBM3E 36 GB 12-high solution brings industry-leading memory technology to AMD Instinct MI350 Series GPU platforms, providing outstanding bandwidth and lower power consumption. The AMD Instinct MI350 Series GPU platforms, built on AMD advanced CDNA 4 architecture, integrate 288 GB of high-bandwidth HBM3E memory capacity, delivering up to 8 TB/s bandwidth for exceptional throughput. This immense memory capacity allows Instinct MI350 series GPUs to efficiently support AI models with up to 520 billion parameters—on a single GPU. In a full platform configuration, Instinct MI350 Series GPUs offers up to 2.3 TB of HBM3E memory and achieves peak theoretical performance of up to 161 PFLOPS at FP4 precision, with leadership energy efficiency and scalability for high-density AI workloads. This tightly integrated architecture, combined with Micron's power-efficient HBM3E, enables exceptional throughput for large language model training, inference and scientific simulation tasks—empowering data centers to scale seamlessly while maximizing compute performance per watt. This joint effort between Micron and AMD has enabled faster time to market for AI solutions.
Return to Keyword Browsing
Nov 18th, 2026 13:00 PST change timezone

New Forum Posts

Popular Reviews

TPU on YouTube

Controversial News Posts