NVIDIA's H100 has recently become available to use via Cloud Service Providers (CSPs), and it was only a matter of time before someone decided to benchmark its performance and compare it to the previous generation's A100 GPU. Today, thanks to the benchmarks of MosaicML, a startup company led by the ex-CEO of Nervana and GM of Artificial Intelligence (AI) at Intel, Naveen Rao, we have some comparison between these two GPUs with a fascinating insight about the cost factor. Firstly, MosaicML has taken Generative Pre-trained Transformer (GPT) models of various sizes and trained them using bfloat16 and FP8 Floating Point precision formats. All training occurred on CoreWeave cloud GPU instances.

Regarding performance, the NVIDIA H100 GPU achieved anywhere from 2.2x to 3.3x speedup. However, an interesting finding emerges when comparing the cost of running these GPUs in the cloud. CoreWeave prices the H100 SXM GPUs at $4.76/hr/GPU, while the A100 80 GB SXM gets $2.21/hr/GPU pricing. While the H100 is 2.2x more expensive, the performance makes it up, resulting in less time to train a model and a lower price for the training process. This inherently makes H100 more attractive for researchers and companies wanting to train Large Language Models (LLMs) and makes choosing the newer GPU more viable, despite the increased cost. Below, you can see tables of comparison between two GPUs in training time, speedup, and cost of training.

Microsoft Corp. today announced the following results for the quarter ended March 31, 2023, as compared to the corresponding period of last fiscal year:

• Revenue was $52.9 billion and increased 7% (up 10% in constant currency)
• Operating income was $22.4 billion and increased 10% (up 15% in constant currency)
• Net income was $18.3 billion and increased 9% (up 14% in constant currency)
• Diluted earnings per share was $2.45 and increased 10% (up 14% in constant currency)

"The world's most advanced AI models are coming together with the world's most universal user interface - natural language - to create a new era of computing," said Satya Nadella, chairman and chief executive officer of Microsoft. "Across the Microsoft Cloud, we are the platform of choice to help customers get the most value out of their digital spend and innovate for this next generation of AI."

According to Ming-Chi Kuo, an industry analyst known for making accurate predictions about Apple, we have some new information regarding Intel's Sapphire Rapids Xeon processors. As Kuo notes, Intel's major Cloud Service Provider (CSP) client, Microsoft, has notified the supply chain that the company is cutting orders of Sapphire Rapids Xeons by 50-70% in the second half of 2023. Interestingly, Intel's supply chain has notified the company to cut chip orders by around 50% amidst weak server demand. This comes straight after Intel's plans to start shipping Sapphire Rapids processors in the second quarter of 2023 and deliver the highly anticipated lineup to customers.

Additionally, Kuo has stated that Intel isn't only competing for clients with AMD but also with Arm-based CPUs. Microsoft also plans to start buying Arm-based server processors made by Ampere Computing in the first half of 2024. This will reduce Microsoft's dependence on x86 architecture and induce higher competition in the market, especially if other CSPs follow.

AMD announced it has joined the Amazon Web Services (AWS) Independent Software Vendor (ISV) Accelerate Program, a co-sell program for AWS Partners - like AMD - who provide integrated solutions on AWS. The program helps AWS Partners drive new business by directly connecting participating ISVs with the AWS Sales organization.

Through the AWS ISV Accelerate Program, AMD will receive focused co-selling support from AWS, including, access to further sales enablement resources, reduced AWS Marketplace listing fees, and incentives for AWS Sales teams. The program will also allow participating ISVs access to millions of active AWS customers globally.

At the NAB 2023 conference, Seagate Technology, a world leader in data storage infrastructure solutions, and QNAP Systems Inc., a leading network attached storage (NAS) vendor, announced their integrated portfolio of edge to cloud enterprise storage solutions. Developed to help small-to-medium sized businesses (SMBs) and content creators manage data from edge to cloud, the portfolio delivers a range of innovative enterprise-scale solutions that include Seagate's IronWolf Pro Hard Drives (HDD), QNAP's high-capacity NAS solutions with Exos E series JBOD systems and Seagate Lyve Cloud.

"Businesses that use data at the edge often rely on NAS devices to store their data. As that data capacity increases and requires optimal protection, businesses are faced with the challenge of backing up data off site," said BS Teh, executive vice president and chief commercial officer at Seagate. "Seagate and QNAP have come together to address this challenge with an entire portfolio of secure mass capacity data solutions to help SMBs address pain points and address the rising costs of data storage and management."

Western Digital has declared that its My Cloud online service has been compromised by a group of hackers late last month: "On March 26, 2023, Western Digital identified a network security incident involving Western Digital's systems. In connection with the ongoing incident, an unauthorized third party gained access to a number of the Company's systems. Upon discovery of the incident, the Company implemented incident response efforts and initiated an investigation with the assistance of leading outside security and forensic experts. This investigation is in its early stages and Western Digital is coordinating with law enforcement authorities."

The statement, issued on April 4, continues: "The Company is implementing proactive measures to secure its business operations including taking systems and services offline and will continue taking additional steps as appropriate. As part of its remediation efforts, Western Digital is actively working to restore impacted infrastructure and services. Based on the investigation to date, the Company believes the unauthorized party obtained certain data from its systems and is working to understand the nature and scope of that data. While Western Digital is focused on remediating this security incident, it has caused and may continue to cause disruption to parts of the Company's business operations."

Today, I'm excited to share how we are expanding Windows 365—your Windows in the cloud—with Windows 365 Frontline. Now in public preview, Windows 365 Frontline helps organizations meet the needs of their entire workforce. We are also delivering Cloud PCs to more devices than ever, with new LG and Motorola integrations.

"Gartner estimates that there are 2.7 billion frontline workers—more than twice the number of desk-based workers." Organizations face unique challenges to meet their IT and workplace experience needs. Companies must scale technology faster and to a much larger population. Equipping all frontline employees with their own devices is not economically feasible for many organizations, particularly when many only require access while on the job. Frontline workers often share physical PCs or kiosks, which some CIOs have told us pose real security and identity challenges. This can inhibit productivity and put critical information like intellectual property or customer data at risk.

On Wednesday, the MLCommons team released the MLPerf 3.0 Inference numbers, and there was an exciting submission from NVIDIA. Reportedly, NVIDIA has used software optimization to improve the already staggering performance of its latest H100 GPU by up to 54%. For reference, NVIDIA's H100 GPU first appeared on MLPerf 2.1 back in September of 2022. In just six months, NVIDIA engineers worked on AI optimizations for the MLPerf 3.0 release to find that basic software optimization can catalyze performance increases anywhere from 7-54%. The workloads for measuring the inferencing speed suite included RNN-T speech recognition, 3D U-Net medical imaging, RetinaNet object detection, ResNet-50 object classification, DLRM recommendation, and BERT 99/99.9% natural language processing.

What is interesting is that NVIDIA's submission is a bit modified. There are open and closed categories that vendors have to compete in, where closed is the mathematical equivalent of a neural network. In contrast, the open category is flexible and allows vendors to submit results based on optimizations for their hardware. The closed submission aims to provide an "apples-to-apples" hardware comparison. Given that NVIDIA opted to use the closed category, performance optimization of other vendors such as Intel and Qualcomm are not accounted for here. Still, it is interesting that optimization can lead to a performance increase of up to 54% in NVIDIA's case with its H100 GPU. Another interesting takeaway is that some comparable hardware, like Qualcomm Cloud AI 100, Intel Xeon Platinum 8480+, and NeuChips's ReccAccel N3000, failed to finish all the workloads. This is shown as "X" on the slides made by NVIDIA, stressing the need for proper ML system software support, which is NVIDIA's strength and an extensive marketing claim.

According to the International Data Corporation (IDC) Worldwide Quarterly Enterprise Infrastructure Tracker: Buyer and Cloud Deployment, spending on compute and storage infrastructure products for cloud deployments, including dedicated and shared IT environments, increased 16.3% year over year in the fourth quarter of 2022 (4Q22) to $24.1 billion. Spending on cloud infrastructure continues to outgrow the non-cloud segment although the latter had strong growth in 4Q22 as well, increasing 9.4% year over year to $18.7 billion. For the full year, cloud infrastructure grew 19.4% to $87.7 billion, while non-cloud grew 13.6% to $66.7 billion. The market continues to benefit from high demand, large backlogs, rising prices, and an improving infrastructure supply chain.

Amazon is today expanding its Luna cloud gaming service into three new territories - Canada, Germany and the United Kingdom. This is the first sign of the online retail giant's goal to broaden the service beyond the initial launch base in the USA. The company is clearly excited to offer their cloud games library to a larger customer base: "Gamers in the U.S. have been enjoying Luna for the past year and we're thrilled to expand the service, giving more customers the opportunity to play high-quality, immersive games without expensive gaming hardware or lengthy downloads."

These new territories have been granted access to Luna's full package which now consists of Ubisoft+, Jackbox Games, and Luna+ subscription services. The Luna app can be launched on 'select devices', which means a wide range of modern bits of kit can run it: Fire TV, Fire Tablets, Windows PCs, Chromebooks, Macs, iPhones, iPads, Android phones and tablets. Amazon confirmed that its Official Luna Wireless Controller is also being made available to customers in Canada, Germany and the United Kingdom - which is an exclusive item to the Amazon Store. It should be noted that you can run other compatible control devices via Bluetooth, including a wireless keyboard and mouse, as well as the Xbox One and PlayStation DualShock 4 gamepads.

Today, Adobe introduced Firefly, a new family of creative generative AI models, first focused on the generation of images and text effects. Firefly will bring even more precision, power, speed and ease directly into Creative Cloud, Document Cloud, Experience Cloud and Adobe Express workflows where content is created and modified. Firefly will be part of a series of new Adobe Sensei generative AI services across Adobe's clouds.Adobe has over a decade-long history of AI innovation, delivering hundreds of intelligent capabilities through Adobe Sensei into applications that hundreds of millions of people rely upon. Features like Neural Filters in Photoshop, Content Aware Fill in After Effects, Attribution AI in Adobe Experience Platform and Liquid Mode in Acrobat empower Adobe customers to create, edit, measure, optimize and review billions of pieces of content with power, precision, speed and ease. These innovations are developed and deployed in alignment with Adobe's AI ethics principles of accountability, responsibility and transparency.

"Generative AI is the next evolution of AI-driven creativity and productivity, transforming the conversation between creator and computer into something more natural, intuitive and powerful," said David Wadhwani, president, Digital Media Business, Adobe. "With Firefly, Adobe will bring generative AI-powered 'creative ingredients' directly into customers' workflows, increasing productivity and creative expression for all creators from high-end creative professionals to the long tail of the creator economy."

TYAN, an industry-leading server platform design manufacturer and a MiTAC Computing Technology Corporation subsidiary, will showcase its latest cloud server platforms powered by AMD EPYC 9004 Series processors and 4th Gen Intel Xeon Scalable processors for next-generation data centers at CloudFest 2023, Booth #H12 in Europa-Park from March 21-23.

"With the exponential advancement of technologies like AI and Machine Learning, data centers require robust hardware and infrastructure to handle complex computations while running AI workloads and processing big data," said Danny Hsu, Vice President of MiTAC Computing Technology Corporation's Server Infrastructure BU. "TYAN's cloud server platforms with storage performance and computing capability can support the ever-increasing demand for computational power and data processing."

AIC today announced its EB202-CP is ready to support newly launched 4th Gen AMD EPYC Embedded 9004 processors. By leveraging the five-year product longevity supported by AMD EPYC Embedded processors, EB202-CP provides customers with stable and long-term support. AIC and AMD will join forces to showcase EB202-CP at Embedded World in AMD stand No. 2-411 from 14th to 16th March, 2023 in Nuremberg, Germany.

AIC EB202-CP, a 2U rackmount server designed for AI and edge appliances, is powered by the newly released 4th Gen AMD EPYC Embedded processors. Featuring the world's highest-performing x86 processor and PCIe 5.0 ready, the 4th Gen AMD EPYC Embedded processors enable low TCO and delivers leadership energy efficiency as well as state-of-the-art security, optimized for workloads across enterprise and edge. EB202-CP, with 22 inch in depth, supports eight front-serviceable and hot-swappable E1.S/ E3.S and four U.2 SSDs. By leveraging the features of the 4th Gen AMD Embedded EPYC processors, EB202-CP is well suited for broadcasting, edge and AI applications, which require greater processing performance and within the most efficient, space-saving format.

Google has made changes to its Google One plans yet again, although this time around, its customers are actually getting more for their money. For those not familiar with the Google One service, it's Google's Dropbox, Microsoft's OneDrive and Apple's iCloud. However, Google has included things like support for its services though its One service and higher tiers have seen added benefits beyond more storage space, such as free access to Google's VPN service. Now the company has decided to give those on its 100 GB US$1.99 and 200 GB US$2.99 service tiers access to not only its VPN service, but has also added what the company calls a dark web report.

The dark web report allows you to search for potential stolen private information on the dark web, or rather, Google does the searching for you and tells you if there are any results in a report. The tricky part here is that you need to have the right info to search for in the first place, but it could be something simple like your email address or phone number, but it could also be your credit card details. The VPN service is for the time being limited to 22 countries that you can find in the screenshot below. It's unlikely that Google offers the best VPN service out there, but it could help protect against man in the middle attacks when using public WiFi if nothing else and it's a decent perk if you're already paying for the Google One service.

For more than a decade, Intel and its partners have been on a mission to virtualize the world's networks, from the core to the RAN (radio access network) and out to the edge, moving them from fixed-function hardware onto programmable, software-defined platforms, making networks more agile while driving down their complexity and cost.

Now operators are looking to cross the next chasm in delivering cloud-native functionality for automating, managing and responding to an increasingly diverse mix of data and services, providing organizations with the intelligence needed at the edge of their operations. Today, Intel announced a range of products and solutions driving this transition and broad industry support from leading operators, original equipment manufacturers (OEMs) and independent software vendors (ISVs).

NVIDIA has recently reported its Q4 earnings, and the earnings call following the report contains exciting details about the company and its plans to open up to new possibilities. NVIDIA's CEO Jensen Huang has stated that the company is on track to become an AI-as-a-Service (AIaaS) provider, which technically makes it a cloud service provider (CSP). "Today, I want to share with you the next level of our business model to help put AI within reach of every enterprise customer. We are partnering with major service -- cloud service providers to offer NVIDIA AI cloud services, offered directly by NVIDIA and through our network of go-to-market partners, and hosted within the world's largest clouds." Said Mr. Huang, adding that "NVIDIA AI as a service offers enterprises easy access to the world's most advanced AI platform, while remaining close to the storage, networking, security and cloud services offered by the world's most advanced clouds. Customers can engage NVIDIA AI cloud services at the AI supercomputer, acceleration library software, or pretrained AI model layers."

In addition to enrolling other CSPs into the race, NVIDIA is also going to offer DGX machines on demand in the cloud. Using select CSPs, you can get access to an entire DGX and harness the computing power for AI research purposes. Mr. Huang noted "NVIDIA DGX is an AI supercomputer, and the blueprint of AI factories being built around the world. AI supercomputers are hard and time-consuming to build. Today, we are announcing the NVIDIA DGX Cloud, the fastest and easiest way to have your own DGX AI supercomputer, just open your browser. NVIDIA DGX Cloud is already available through Oracle Cloud Infrastructure and Microsoft Azure, Google GCP, and others on the way."

Marvell Technology, Inc., a leader in data infrastructure semiconductor solutions, announced today that it has selected Amazon Web Services, Inc. (AWS) as its cloud provider for electronic design automation (EDA). A cloud-first approach helps Marvell to rapidly and securely scale its service on the world's leading cloud, rise to the challenges brought by increasingly complex chip design processes, and deliver continuous innovation for the expanding needs across the automotive, carrier, data center, and enterprise infrastructure markets it serves. The work extends the longstanding relationship between the two companies—Marvell is also a key semiconductor supplier for AWS, helping the company support the design and rapid delivery of cloud services that best meet customers' demanding requirements.

EDA refers to the specialized and compute-intensive processes used in chip making and is a critical piece of Marvell's R&D. Over the years, the number of transistors on an integrated chip has increased exponentially. Each advance in chip design calls for a calculated application of software modules overseeing logic design, debugging, component placement, wire routing, optimization of time and power consumption, and verification. Due to the computationally intensive nature of EDA workloads, it is no longer cost-effective or timely to run EDA on premises. By powering its EDA with AWS, Marvell leverages an unmatched portfolio of services including secure, elastic, high-performance compute capacity in the cloud to solve challenges around speed, latency, security of IP, and data transfer.

Facing global economic headwinds, the four major North American cloud service providers (CSPs) have scaled back their server procurement quantities for 2023 and could make further downward corrections in the future. Meta is the leader among the four in terms of server demand reduction, followed by Microsoft, Google, and AWS. TrendForce has lowered the YoY growth rate of their total server procurement quantity for this year from the original projection of 6.9% to the latest projection of 4.4%. With CSPs cutting demand, global server shipments are now estimated to grow by just 1.87% YoY for 2023. Regarding the server DRAM market, prices there are estimated to drop by around 20~25% QoQ for 1Q23 as CSPs' downward corrections exacerbate the oversupply situation.

Looking at the four CSPs individually, the YoY decline of Meta's server procurement quantity has been widened to 3.0% and could get larger. The instability of the global economy remains the largest variable for all CSPs. Besides this, Meta has also encountered a notable obstacle in expanding its operation in Europe. Specifically, its data center in Denmark has not met the regional standard for emissions. This issue is expected to hinder its progress in setting up additional data centers across the EU. Moreover, businesses related to e-commerce account for about 98% of Meta's revenue. Therefore, the decline in e-commerce activities amidst the recent easing of the COVID-19 pandemic has impacted Meta's growth momentum. Additionally, Meta's server demand has been affected by the high level of component inventory held by server ODMs.

Intel Corporation today reported fourth-quarter and full-year 2022 financial results. The company also announced that its board of directors has declared a quarterly dividend of $0.365 per share on the company's common stock, which will be payable on March 1, 2023, to shareholders of record as of February 7, 2023.

"Despite the economic and market headwinds, we continued to make good progress on our strategic transformation in Q4, including advancing our product roadmap and improving our operational structure and processes to drive efficiencies while delivering at the low-end of our guided range," said Pat Gelsinger, Intel CEO. "In 2023, we will continue to navigate the short-term challenges while striving to meet our long-term commitments, including delivering leadership products anchored on open and secure platforms, powered by at-scale manufacturing and supercharged by our incredible team."

Dell Technologies expands the industry's top selling server portfolio, with an additional 13 next-generation Dell PowerEdge servers, designed to accelerate performance and reliability for powerful computing across core data centers, large-scale public clouds and edge locations. Next-generation rack, tower and multi-node PowerEdge servers, with 4th Gen Intel Xeon Scalable processors, include Dell software and engineering advancements, such as a new Smart Flow design, to improve energy and cost efficiency. Expanded Dell APEX capabilities will help organizations take an as-a-Service approach, allowing for more effective IT operations that make the most of compute resources while minimizing risk.

"Customers come to Dell for easily managed yet sophisticated and efficient servers with advanced capabilities to power their business-critical workloads," said Jeff Boudreau, president and general manager, Infrastructure Solutions Group, Dell Technologies. "Our next-generation Dell PowerEdge servers offer unmatched innovation that raises the bar in power efficiency, performance and reliability while simplifying how customers can implement a Zero Trust approach for greater security throughout their IT environments."

IceWhale Technologies has launched ZimaBoard, a low-cost single board server that functions as personal cloud storage and software router, and allows users to set up a VPN, stream 4K video, access media collections, and reinvent smart home projects on Amazon. Three models with tiered processing power are available in the brand's shop for US customers: ZimaBoard 216 (2 GB RAM, 16 GB eMMC storage, dual-core Intel Celeron processor) is $139.99 USD, ZimaBoard 432 (4 GB RAM, 32 GB eMMC storage, quad-core Intel Celeron processor) is $169.99 USD, and ZimaBoad 832 (8 GB RAM, 32 GB eMMC storage, quad-core Intel Celeron processor) is $229.99 USD.

"Following our Kickstarter campaign and fulfillment to backers in July, we're excited to announce our move to Amazon," said founder Lauren Pan. "Getting here was no small feat, especially after more than a year of chip shortages and unpredictable lockdowns, but here we are. ZimaBoard is one board to rule them all—one that puts individuals back in control."

Innodisk, a leading global provider of industrial-grade flash storage, DRAM memory and embedded peripherals, has announced its latest step into the AI market, with the launch of EXMU-X261, an FPGA Machine Vision Platform. Powered by AMD's Xilinx Kria K26 SOM, which was designed to enable smart city and smart factory applications, Innodisk's FPGA Machine Vision Platform is set to lead the way for industrial system integrators looking to develop machine vision applications.

Automated defect inspection, a key machine vision application, is an essential technology in modern manufacturing. Automated visual inspection guarantees that the product works as expected and meets specifications. In these cases, it is vital that a fast and highly accurate inspection system is used. Without AI, operators must manually inspect each product, taking an average of three seconds per item. Now, with the help of AI solutions such as Innodisk's FPGA Machine Vision Platform, product inspection in factories can be automated, and the end result is not only faster and cheaper, but can be completely free of human error.

During the Snapdragon Summit 2022, Qualcomm Technologies, Inc. and Adobe announced an expanded collaboration to support creative experiences on Snapdragon powered devices across mobile, compute, and XR platforms. Together, the companies will push the boundaries of creativity and document productivity for Snapdragon users.

Building on the rich experiences already found with Adobe Photoshop and Adobe Lightroom for Windows on Snapdragon devices, Adobe remains committed to bringing the best of Creative Cloud natively to the Snapdragon platform, including Adobe Fresco and Adobe Acrobat. Bringing the strength of Adobe Sensei, Adobe's artificial intelligence (AI) technologies, alongside the dedicated on-device Qualcomm AI Engine on Snapdragon compute platforms, users can experience enhanced device performance for creative workflows and tasks including super-resolution, background removal and color enhancement, editing content on-the-go, and more.

HyperX, the gaming peripherals team at HP Inc. and brand leader in gaming and esports, today announced the release of the HyperX Cloud Stinger 2 wireless gaming headset. Offering signature HyperX comfort and durability with soft memory foam and premium leatherette, the Cloud Stinger 2 wireless includes DTS Headphone:X Spatial Audio for improved 3D audio spatialization and localization accuracy and up to 20 hours of battery life for cable-free gaming.

Cloud Stinger 2 wireless utilizes large 50 mm drivers to provide clearer audio with DTS Headphone:X virtual spatial audio to deliver an immersive 7.1 sound environment. The headset also features easy-access audio controls on the earcup and a swivel-to-mute, noise-canceling flexible microphone for added convenience, reduced background noise and clearer voice quality.

Rescale, the leader in high performance computing built for the cloud to accelerate engineering innovation, today announced it is teaming with NVIDIA to integrate the NVIDIA AI platform into Rescale's HPC-as-a-Service offering. The integration is designed to advance computational engineering simulation with AI and machine learning, helping enterprises commercialize new product innovations faster, more efficiently and at less cost.

Additionally, Rescale announced the world's first Compute Recommendation Engine (CRE) to power Intelligent Computing for HPC and AI workloads. Optimizing workload performance can be prohibitively complex as organizations seek to balance decisions among architectures, geographic regions, price points, scalability, service levels, compliance, and sustainability objectives. Developed using machine learning on NVIDIA architectures with infrastructure telemetry, industry benchmarks, and full-stack metadata spanning over 100 million production HPC workloads, Rescale CRE provides customers unprecedented insight to optimize overall performance.