Red Hat & AMD Strengthen Strategic Collaboration - Leading to More Efficient GenAI

[T0@st]

Red Hat, the world's leading provider of open source solutions, and AMD today announced a strategic collaboration to propel AI capabilities and optimize virtualized infrastructure. With this deepened alliance, Red Hat and AMD will expand customer choice across the hybrid cloud, from deploying optimized, efficient AI models to more cost-effectively modernizing traditional virtual machines (VMs). As workload demand and diversity continue to rise with the introduction of AI, organizations must have the capacity and resources to meet these escalating requirements. The average datacenter, however, is dedicated primarily to traditional IT systems, leaving little room to support intensive workloads such as AI. To answer this need, Red Hat and AMD are bringing together the power of Red Hat's industry-leading open source solutions with the comprehensive portfolio of AMD high-performance computing architectures.

AMD and Red Hat: Driving to more efficient generative AI
Red Hat and AMD are combining the power of Red Hat AI with the AMD portfolio of x86-based processors and GPU architectures to support optimized, cost-efficient and production-ready environments for AI-enabled workloads. AMD Instinct GPUs are now fully enabled on Red Hat OpenShift AI, empowering customers with the high-performing processing power necessary for AI deployments across the hybrid cloud without extreme resource requirements. In addition, using AMD Instinct MI300X GPUs with Red Hat Enterprise Linux AI, Red Hat and AMD conducted testing on Microsoft Azure ND MI300X v5 to successfully demonstrate AI inferencing for scaling small language models (SLMs) as well as large language models (LLM) deployed across multiple GPUs on a single VM, reducing the need to deploy across multiple VMs and reducing performance costs.




To drive further performance acceleration and tuning capabilities, Red Hat and AMD are collaborating in the upstream vLLM community to foster more efficient AI inference. With this upstream activation, Red Hat and AMD aim to deliver:

• Improved performance on AMD GPUs: By upstreaming the AMD kernel library and optimizing various components like the Triton kernel and FP8, Red Hat and AMD are advancing inference performance for both dense and quantized models, enabling faster and more efficient execution of vLLM on AMD Instinct MI300X accelerators.
• Enhanced multi-GPU support: Improving collective communication and optimizing multi-GPU workloads opens the door to more scalable and energy-efficient AI deployments, which is particularly beneficial for workloads that require distributed computing across multiple GPUs, reducing bottlenecks and improving overall throughput.
• Expanded vLLM ecosystem engagement: Cross-collaboration between Red Hat, AMD and other industry leaders like IBM helps accelerate upstream development to propel continuous improvements for both the vLLM project and AMD GPU optimization, further benefiting vLLM users that rely on AMD hardware for AI inference and training.

Building upon this collaboration in the vLLM community, AMD Instinct GPUs will support Red Hat AI Inference Server, Red Hat's enterprise-grade distribution of vLLM, out-of-the-box for a powerful, reliable and scalable AI inference server. As the top commercial contributor to vLLM, Red Hat is committed to enabling compatibility when deploying vLLM on an organization's hardware of choice, which includes AMD Instinct GPUs. Running vLLM on AMD Instinct GPUs empowers organizations to deploy any open source AI model on validated, tested GPU hardware for outstanding optimization and performance.

AMD EPYC CPUs also enable end-to-end AI performance and are ideal to host GPU-enabled systems. This can help improve the performance and return on investment (ROI) of each GPU server for even the most demanding of AI workloads.

Transforming the modern datacenter
By optimizing existing datacenter footprints, organizations can more effectively and easily reinvest resources to enable AI innovation. Red Hat OpenShift Virtualization, a feature of Red Hat OpenShift, offers a streamlined path for organizations to migrate and manage VM workloads with the simplicity and speed of a cloud-native application platform. Red Hat OpenShift Virtualization is validated for AMD EPYC processors capable of leveraging the AMD EPYC processors' excellent performance and power efficiency, wherever needed on the hybrid cloud, while maintaining a bridge to a cloud-native future.



Red Hat OpenShift Virtualization on AMD EPYC CPUs helps enterprises optimize application deployment on leading servers, such as Dell PowerEdge, HPE ProLiant and Lenovo ThinkSystem products. When refreshing a legacy datacenter, Red Hat OpenShift Virtualization provides for the unification of VMs and containerized applications, on-premise, in public clouds or across the hybrid cloud. This helps enable high infrastructure consolidation ratios that can lead to significantly lower total cost of ownership (TCO) on hardware, software licensing and energy dimensions. This has the added benefit of enabling IT teams to more effectively manage critical workloads today while freeing resources and energy to apply to AI workloads now and in the future.

View at TechPowerUp Main Site | Source

 

[dir_d]

Very smart for AMD to partner with Red Hat. Hopefully this helps when UDNA comes out, the benefits could trickle down into consumer distros. Who knows, this could make FSR 4 more efficient if they continue FP8 improvements.

 

[snapdragon]