Thursday, March 17th 2022
Apple's Graphics Performance Claims Proven Exaggerated by Mac Studio Reviews
Apple made some bold claims at the launch of its new Mac Studio computers when it came to the performance of the new systems and it looks like Apple was exaggerating those claims by quite some margin when it comes to the graphics performance. The first reviews of the new Mac Studio went live today and thanks to those reviews, despite the limited benchmarks that were performed on the new systems from Apple, that as so often Apple's performance metrics are still relying on the reality distortion field. Most of the publications that got their hands on the new systems focused on CPU benchmarks and there's no doubt the Ultra version of the M1 processor is a beast when used for things like video rendering and complex image manipulation, where it's butting heads with an AMD Ryzen Threadripper 3970X.
However, Apple's 64-core GPU isn't quite what the company claimed. In the presentation footnotes Apple provided details on the "highest-end discrete GPU" that they compared to, which was an NVIDIA GeForce RTX 3090. If we were to be kind to Apple, we would say that the company was slightly off target here, but it's actually not even remotely close. Tom's Guide tested the M1 Ultra SoC in Sid Meier's Civilization 6 and got a whopping 38.85 FPS at 1440p, which is beaten soundly by a Lenovo ThinkPad P1 Gen 4 with a GeForce RTX 3070 laptop GPU that scored 64.9 FPS at 4K. Likewise, The Verge decided to test the claims and had a system with an actual GeForce RTX 3090 in it and ran Shadow of the Tomb Raider and the PC managed 142 FPS at 1080p, with the M1 Ultra coming in quite far behind at 108 FPS. Moving up to 1440p the 3090 came in at 114 FPS, with the M1 still trailing behind, if not quite as badly at 96 FPS.The issue here isn't the actual performance, as it's no doubt decent figures considering the max power draw of the Mac Studio is 370 W, far less than any PC with a GeForce RTX 3090 in it. However, Apple's inflated performance claims are getting old and tired and the company really needs to do better. It's highly unlikely that anyone will be buying a US$4999 Mac Studio to play games on, as that's what the base model with the 64-core GPU costs. As such, Apple should provide realistic benchmark comparisons, as overall the Mac Studio is an impressive piece of kit in its own right. Based on the first reviews, it's likely going to end up being used by a lot of creative companies that would normally use Macs for their workflow.
Sources:
Tom's Guide, The Verge
However, Apple's 64-core GPU isn't quite what the company claimed. In the presentation footnotes Apple provided details on the "highest-end discrete GPU" that they compared to, which was an NVIDIA GeForce RTX 3090. If we were to be kind to Apple, we would say that the company was slightly off target here, but it's actually not even remotely close. Tom's Guide tested the M1 Ultra SoC in Sid Meier's Civilization 6 and got a whopping 38.85 FPS at 1440p, which is beaten soundly by a Lenovo ThinkPad P1 Gen 4 with a GeForce RTX 3070 laptop GPU that scored 64.9 FPS at 4K. Likewise, The Verge decided to test the claims and had a system with an actual GeForce RTX 3090 in it and ran Shadow of the Tomb Raider and the PC managed 142 FPS at 1080p, with the M1 Ultra coming in quite far behind at 108 FPS. Moving up to 1440p the 3090 came in at 114 FPS, with the M1 still trailing behind, if not quite as badly at 96 FPS.The issue here isn't the actual performance, as it's no doubt decent figures considering the max power draw of the Mac Studio is 370 W, far less than any PC with a GeForce RTX 3090 in it. However, Apple's inflated performance claims are getting old and tired and the company really needs to do better. It's highly unlikely that anyone will be buying a US$4999 Mac Studio to play games on, as that's what the base model with the 64-core GPU costs. As such, Apple should provide realistic benchmark comparisons, as overall the Mac Studio is an impressive piece of kit in its own right. Based on the first reviews, it's likely going to end up being used by a lot of creative companies that would normally use Macs for their workflow.


151 Comments on Apple's Graphics Performance Claims Proven Exaggerated by Mac Studio Reviews
In terms of CPU power the m1 ultra is still competing - and losing, with mainstream x86 cpus from last gen. Meteorlake and zen 4 is due to release soon that will be on a completely different level as well. Saying that apple competes with the top of the stack is just a joke, it competes with the mainstream parts, and it doesn't even beat those
Do not start insulting... follow the guidelines/rules.
So exactly what, are you on about what point was made.
If you want to compare it to some proper equipment, compare it to an A 6000 at fp64. Its how many times slower, 20? More than 20?
Also worth noting: AT measures per-core memory access speeds for the M1 to be significantly lower than the total bandwidth of the memory interface. This is also true for other architectures (you need more than 1c to max out your bandwidth), but it makes direct bandwidth comparisons troublesome.
Still, this tells us several things:
- The Apple Firestorm cores (1t) are ever so slightly behind AMD's Zen 3 cores (2t) in integer workloads, but are almost matched core for core.
- The Apple Firestorm cores have a massive advantage over Zen3 in floating point workloads, at least those present in the SPEC suite, delivering more than 2x the performance per core. This has the caveat that these workloads are more memory bound.
- nT scaling is quite different across the architectures and workloads: the 5950X scales 10.87x from 1t to 32t in int and 5.3x in fp. The 5800X is sadly not in the 1t chart, but should be marginally slower than the 5950X (2-300MHz). If they were identical, its scaling would be 6.7x and 3.9x from 1t to 16t, so it's likely a tad higher than that. The M1 Max scales by 7.1x and 6.3x from 1t to 10t. Thread scaling comparisons are made difficult by Apple's big.little architecture and AMD's SMT, but it does seem that the higher memory bandwidth helps Apple scale better with additional cores to some extent (though this could also be affected by many other factors, including software). Or this could be formulated as AMD's MSDT platform being significantly held back by memory bandwidth.
How does this affect the discussions here? Well, it's well documented that Cinebench doesn't care much about memory bandwidth. It likes higher IF and memory clocks on AMD, but it scales poorly with more memory channels, meaning latency is more important than bandwidth. Other workloads are quite different - but they tend to be quite specialized. There are essentially no consumer workloads that are particularly bandwidth limited, with most caring more about compute power or latency.
Does this mean Cinebench is a more, or less, reliable benchmark? Depends what you're looking for. It's clear that Apple has designed the larger M1 chips for bandwidth-hungry applications (though the large GPU sharing it also needs that, of course). So, if your workloads are bound by memory bandwidth, the M1 Ultra and its siblings are likely to deliver staggeringly good performance and efficiency. If not? Then it's likely competitive with competing CPUs with a similar number of cores (not threads), but YMMV depending on the workload.
Does this mean Apple lied? Again, no. There's no doubt that the M1 Ultra is the most powerful chip for PCs ever made. Not CPU, not GPU, but in sum.Being a single SoC has some great advantages, particularly in efficiency (near zero interconnect power, far less embodied energy in duplicate componentry), but also to some extent in latency-sensitive performance scenarios. But as you say, it's also inflexible, and those advantages don't necessarily apply to all workloads. Both approaches have distinct pros and cons. You can't get past the fact that a tightly integrated package will always be more efficient than a collection of discrete components (as long as they are othewise comparable in terms of architectural efficiency). There's a reason why a 5W smartphone delivers a lot more than 1/20th the performance of a 100W laptop. That doesn't invalidate the value or performance of the laptop - after all, it delivers a degree of performance impossible in a smartphone.
As for that fp64 comparison: according to this source the M1 GPU arch has 1/4 speed fp64, so it should deliver ~2.6TF/s of FP64 (assuming 10.4TF FP32 numbers online are accurate). For comparison, Ampere has 1/32 speed FP64, and delivers ~1.21TF/s of FP64. Not that this necessarily matters much: the reason Ampere performs poorly in FP64 is that double precision floating point math has become ever more of a niche application, and Nvidia thus doesn't prioritize it whatsoever outside of their datacenter GA100 chips, which have 1:2 FP64 (and the A100 80GB SXM4 delivers ~9.746TF/s of FP64). As such, it seems that Apple cares more about FP64 than Nvidia does outside of datacenters, likely indicating that pro applications common on MacOS tend to use that a bit more than most PC applications (though it might also just be a small but profitable subset of apps, wanting to cater to a niche that pays well).
It's possible they used some form of benchmark that could partially make use of the ME - say, a video editing timeline performance benchmark with various filters applied, where the video playback itself is accelerated through the ME - but that's rather unlikely (and still places most of the workload on the GPU, not the ME).
We still don't have sufficient in-depth benchmarks to really conclude about anything - and there seems to be a lack of good cross-platform benchmarks in general, which makes comparisons difficult. But from what we've seen so far, Apple's framing seems like typical PR: grounded in some specific truth, but highly specific and borderline misleading unless you really pay attention. Their headline statements are still true - this is the most powerful chip ever in a PC - but only when you pay attention to what they're actually saying, rather than what they are (very strongly) insinuating. Which just tells us that Apple has a very good marketing team.
I'm sure a well tuned 3090 could bring these power comparisons at iso performance much closer, but I kind of doubt they'd be able to overtake Apple - though making these cross-architectural comparisons without anything resembling good benchmarking at these power levels is of course a major challenge, rendering this just loose speculation. Still, Nvidia is on a relatively inefficient node, while Apple uses the best available, and a low power version of it to boot. It's not quite the same as the 5nm node other mobile customers use, but they're clearly not using high performance libraries here - clocks are too low and density is too high for that. The question then becomes whether Apple's architecture is sufficiently behind Nvidia's to nullify the effects of their near 2-node advantage.
Of course that's also a part of why Nvidia is struggling with efficiency this generation - being stuck on an underperforming node isn't helping them compete with a resurgent AMD, who used to be miles behind in efficiency but is now beating them. AMD also has a node advantage of course, but the efficiency improvements of RDNA and RDNA2 seem to have caught Nvidia somewhat unaware, forcing them to push clocks higher than what their node can really comfortably handle.
As for area and transistor count, it's of course difficult to tell just how large the GPU (and other parts matching hardware present on an Nvidia GPU, especially once you start factoring in memory interfaces, caches, etc.) on this is, but it seems reasonable to estimate that it (including memory interfaces and caches) covers ~50% of the die area, or a bit less. Accounting for the density advantage of TSMC's custom Apple 5nm process vs. Samsung 8nm, that likely means the GPU is comparable to the 628mm² GA102 - though that doesn't really add up in terms of transistor counts, seeing how even even the M1 Max has 2x the transistor count of GA102. Of course transistor counts discussed on this level are kind of arbitrary, and mainly speak to architectural and node differences, but it still seems reasonable to assume that the M1 Ultra has a GPU that hardware-wise is in the same ballpark as the 3090 - but clocked much lower.
The M1 Max was rated at 10.4TF/s FP32, and assuming the same clock speeds on the Ultra, that's a 20.8TF/s GPU - that's half the FP32 performance estimated for the 3090 Ti, but that's also a bit of a skewed comparison given how consumer Ampere doubled its theoretical FP32 output through implementing dual-mode ALUs. Which also means that you don't always get that level of performance, but anything from half to full depending on the composition of your workload. Of course we don't really know anything about Apple's GPU architecture, but given that it originated from an Imagination design, and the fact that it has a relatively high FP64-to-FP32 ratio, it's pretty safe to assume it is overall more similar to pre-Ampere Nvidia architectures or the GA100. (For reference, the compute-focused GA100 doesn't have the dual-mode ALUs and thus seemingly has fewer shading units despite being a much larger GPU.)
Given all of that, you're probably right that this is comparable to a low-clocked 3090 in many ways (though on a much more efficent node). Though given the quirks of architecture, OS, APIs and software, this comparison will likely skew wildly across various workloads.