Thursday, March 17th 2022
Apple's Graphics Performance Claims Proven Exaggerated by Mac Studio Reviews
Apple made some bold claims at the launch of its new Mac Studio computers when it came to the performance of the new systems and it looks like Apple was exaggerating those claims by quite some margin when it comes to the graphics performance. The first reviews of the new Mac Studio went live today and thanks to those reviews, despite the limited benchmarks that were performed on the new systems from Apple, that as so often Apple's performance metrics are still relying on the reality distortion field. Most of the publications that got their hands on the new systems focused on CPU benchmarks and there's no doubt the Ultra version of the M1 processor is a beast when used for things like video rendering and complex image manipulation, where it's butting heads with an AMD Ryzen Threadripper 3970X.
However, Apple's 64-core GPU isn't quite what the company claimed. In the presentation footnotes Apple provided details on the "highest-end discrete GPU" that they compared to, which was an NVIDIA GeForce RTX 3090. If we were to be kind to Apple, we would say that the company was slightly off target here, but it's actually not even remotely close. Tom's Guide tested the M1 Ultra SoC in Sid Meier's Civilization 6 and got a whopping 38.85 FPS at 1440p, which is beaten soundly by a Lenovo ThinkPad P1 Gen 4 with a GeForce RTX 3070 laptop GPU that scored 64.9 FPS at 4K. Likewise, The Verge decided to test the claims and had a system with an actual GeForce RTX 3090 in it and ran Shadow of the Tomb Raider and the PC managed 142 FPS at 1080p, with the M1 Ultra coming in quite far behind at 108 FPS. Moving up to 1440p the 3090 came in at 114 FPS, with the M1 still trailing behind, if not quite as badly at 96 FPS.The issue here isn't the actual performance, as it's no doubt decent figures considering the max power draw of the Mac Studio is 370 W, far less than any PC with a GeForce RTX 3090 in it. However, Apple's inflated performance claims are getting old and tired and the company really needs to do better. It's highly unlikely that anyone will be buying a US$4999 Mac Studio to play games on, as that's what the base model with the 64-core GPU costs. As such, Apple should provide realistic benchmark comparisons, as overall the Mac Studio is an impressive piece of kit in its own right. Based on the first reviews, it's likely going to end up being used by a lot of creative companies that would normally use Macs for their workflow.
Sources:
Tom's Guide, The Verge
However, Apple's 64-core GPU isn't quite what the company claimed. In the presentation footnotes Apple provided details on the "highest-end discrete GPU" that they compared to, which was an NVIDIA GeForce RTX 3090. If we were to be kind to Apple, we would say that the company was slightly off target here, but it's actually not even remotely close. Tom's Guide tested the M1 Ultra SoC in Sid Meier's Civilization 6 and got a whopping 38.85 FPS at 1440p, which is beaten soundly by a Lenovo ThinkPad P1 Gen 4 with a GeForce RTX 3070 laptop GPU that scored 64.9 FPS at 4K. Likewise, The Verge decided to test the claims and had a system with an actual GeForce RTX 3090 in it and ran Shadow of the Tomb Raider and the PC managed 142 FPS at 1080p, with the M1 Ultra coming in quite far behind at 108 FPS. Moving up to 1440p the 3090 came in at 114 FPS, with the M1 still trailing behind, if not quite as badly at 96 FPS.The issue here isn't the actual performance, as it's no doubt decent figures considering the max power draw of the Mac Studio is 370 W, far less than any PC with a GeForce RTX 3090 in it. However, Apple's inflated performance claims are getting old and tired and the company really needs to do better. It's highly unlikely that anyone will be buying a US$4999 Mac Studio to play games on, as that's what the base model with the 64-core GPU costs. As such, Apple should provide realistic benchmark comparisons, as overall the Mac Studio is an impressive piece of kit in its own right. Based on the first reviews, it's likely going to end up being used by a lot of creative companies that would normally use Macs for their workflow.


151 Comments on Apple's Graphics Performance Claims Proven Exaggerated by Mac Studio Reviews
Then for some reason on the SSD storage for new MAC's then don't seem to have controllers or anything on them. Seems like Apple just is shortcutting things on the storage side of things, which seems kind of important if you care for your data.
"Hector Martin works on Asahi Linux for AppleSilicon Macs:
"Well, this is unfortunate. It turns out Apple's custom NVMe drives are amazingly fast - if you don't care about data integrity. If you do, they drop down to HDD performance. Thread."
"For a while, we've noticed that random write performance with fsync (data integrity) on Asahi Linux (and also on Linux on T2 Macs) was terrible. As in 46 IOPS terrible. That's slower than many modern HDDs. We thought we were missing something, since this didn't happen on macOS.""
API's and different things matter. It is like when Doom came out and supported Vulkan. The FPS jumped up drastically especially for AMD. That is why context matters. Hardware is made to run certain API's etc and Apple shows their hardware in the best light. If you are going to do that you have to take synthetic benchmarks out and then run the hardware you are testing them against in their best case scenarios.
Like here for instance is an Nvidia graphic:
M1 Ultra Mac Studio - Benchmarks & Thermals (The TRUTH!) - YouTube
The Mac Studio is not what many people wanted: Apple going all out. It's just a doubled up M1 Max, a laptop chip. Because they are so far ahead of the competition they are holding back. Where is the 4ghz+ CPU chip? The one that actually uses 200W? And you can also see anomalies that show that Mac OS software hasn't been written for the new chip yet, such as a video export being the same speed as with the Max chip. But you buy a product the way it works today, not tomorrow. Apple doesn't want leaks, they didn't seed the product to developers in advance. Wait a month and look at it again. Also Anandtech and Maxtech and perhaps HardwareUnboxed are the only ones doing decent work. You can watch 100 videos on youtube, most of those guys know nothing about computers.
In his tests look at the actual CPU and GPU power usage, not even a fraction of what the M1 Ultra is even capable of using. Software issues. And then even at full load it is only a fraction of what the chip could do if we had unlocked power and voltage and clock speeds. It's running at only 3.0 Ghz, not even 3.2 Ghz.
I'm not sure it is safe to say we are just seeing a fraction of what M1 Ultra can do because they wouldn't of added the cost of a second chip if they could just up the power and speed of the chip. That wouldn't make sense for them financially. So there definitely is a limitation they have run into there. At least at the moment. No company spends extra money when they don't have to.
It all really makes sense if taken as a whole.
videocardz.com/newz/apple-m1-ultra-package-pictured-nearly-3-times-bigger-than-amd-ryzen-cpu
To compare the PERFORMANCE, SIZE and COST of the M1 Ultra you need:
1) A CPU from INTEL or AMD
2) A GPU form AMD or NIVIDA
3) 128GB of RAM
So now go back to that photo and add in all of the missing components, a GPU and RAM. Love to see the total COST when that's done. Throw in a motherboard to socket the RAM too.
128gb memory $600
3970x threadripper $2500
sTRX4 motherboard $500
1000w psu gold rated $180
3080ti $1300
atx case $200
1tb nvme $100
cost a little over $5400 (compared to M1 Studio with 128gb of memory at $5800) . But technically you at this point have more memory because the memory from Apple is unified. So in the PC build you would have 128gb on the motherboard plus 12gb dedicated to the GPU for a total of 140gb. The threadripper is 32 cores 64 threads so you can go one step down to the 3960x with 24 cores 48 threads for $700 less and get the price down to $4700.
It might not be more energy efficient and run as cool but that would be a pretty powerful machine. The 3970x gets you pretty close to the geekbench score of the M1 ultra. about 100 points lower. Not sure how the threadripper overclocks.
The pc build has the ability to add larger nvme's down the road more memory etc because it is upgradeable.
If your dropped the memory down to 64gb or 32gb's though there would be massive savings on the pc side because you can get 32gb's for under $150. Apple has a huge price increase for memory from 32gb to 64gb is $400 increase and to 128gb the price goes up $1200.
Also, if MacOS was based around "a single API" as you say ... that wouldn't help much. The API would still need the same breadth of features and functionality of whatever combination of APIs competitors deal with. Integrating those features into a single API doesn't necessarily simplify things all that much. What matters is how well things are designed to interface with each other and how easy they are to learn and to program for. There is also nothing stopping AMD designing a Metal implementation of TressFX, for example - APIs stack on top of each other.Yeah, I'm well aware of this. If you've paid attention, you'd also notice that I have at no point extolled Apple's storage as anything special - in part because of this. Though saying "their SSDs don't seem to have controllers on them" is a misunderstanding - Apple has a very different storage hardware architecture than PCs, with the SSD controller integrated into the T2 system controller (which also handles security, encryption, etc.). In other words, the SSD controller doesn't live on the SSD. This is a really weird choice IMO, and while I see its benefits (lower cost, less duplicate hardware in the system), I think it overall harms the system's performance.Yes, it's obvious that APIs matter a lot, though your representation of this is a bit weird. It's not like AMD's hardware is inherently designed to be faster in Vulkan than DX12, it's just a quirk of their architecture compared to Nvidia. It wouldn't make sense for them to make such a design consciously, as Vulkan is far rarer than DX11/12, and isn't used at all in consoles, which is a massive focus for AMD GPU architectures. Still, it is indeed important to keep in mind that different architectures interface differently with different APIs and thus perform differently - and that any given piece of software, especially across platforms, can use many different APIs to do "the same" work. That Nvidia graph is indeed a good example, as part of the background behind that is that most of those apps are not Metal native - Blender Cycles, for example, got its Metal render backend 11 days ago, while it has had support for highly accelerated workloads on Nvidia with first CUDA, then the Optix renderer, for several years.
This just illustrates the difficulty of doing reasonable cross-platfrom benchmarks, and the very real discussion of whether one should focus on real-world applications with all their specific quirks (the Nvidia graph above was true at the time, after all, with those applications), or if the goal is to demonstrate the actual capabilities of the hardware on as level a playing field as possible. Platform-exclusive software makes this even more complicated, obviously.This is rather inaccurate. While early ARM designs were indeed focused only on low power, there has been a concerted effort for high-performance ARM chips for more than half a decade, including server and datacenter chips from a variety of vendors. Apple is by no means alone in this - though their R&D resources are clearly second to none. Also, AFAIK, Apple can't add instruction sets - their chips are based on the ARMv8.4-A instruction set. And while I'm reasonably sure there are architectural reasons for the M1 not scaling past ~3.2 GHz, it's impossible from an end user point of view to separate those from other hardware properties: the specific production node (Apple uses low-power, high density mobile-oriented nodes for all chips, with competitors like AMD and Intel using high-clocking nodes instead); their chips are designed with extremely wide execution pipelines, far beyond any other ARM (or x86) design, which makes clocking them high potentially problematic. You can't unilaterally peg the performance ceiling of the M1 family on it being ARM-based.
As for the R&D and attributed costs: Apple has been developing their own core and interconnect designs since the A6 in 2012. Their ARM licence is for fully custom designed cores, meaning they do not base their designs on ARM designs at all, and have not done so for a decade now.This is problematic on many, many levels. First off: All modern CPUs have cache. GPUs also have cache, though in very varying amounts. Bringing that up as a difference between an SoC and a CPU is ... meaningless. (As is the distinction between SoC and CPU today - all modern CPUs from major vendors are SoCs, but with differing featuresets.) As for one stick of DDR5 having more transistors, that may be true, but that stick also has 8 or more dice on board, and those cosist nearly exclusively of a single type of high density transistor made on a bespoke, specific-purpose node. Logic transistors are far more complex and take much more space than memory transistors, and logic nodes are inherently less dense than memory and cache nodes (illustrated by how the cache die on the Ryzen 7 5800X3D fits 64MB of cache into the same area as the 32MB of cache on the CCD). If comparing transistor counts, the relevant comparisons must be of reasonably similar silicon, i.e. complex and advanced logic chips.
And no, we don't have feature-by-feature transistor counts. But that doesn't ultimately matter, as we can still combine the transistor counts of competing solutions for a reasonable approximate comparison - and Apple is way above anything else even remotely comparable. This illustrates that they are achieving the combination of performance and efficiency they are by going all-in on a "wide-and-slow" design at every level (as Anandtech notes in their M1 coverage, the execution pipeline of the M1 architecture is also unprecedentedly wide for a consumer chip). This is clearly a conscious choice on their part, and one that can likely in part be attributed to their vertical integration - they don't need to consider per-chip pricing or profit margins (which for Intel and AMD tend to be ~40%), but can have a more holistic view, integrating R&D costs and production costs into the costs of the final hardware. It is far less feasible for a competitor to produce anything comparable just because of the sheer impossibility of getting any OEM to buy the chip to make a product with it.I mean, I mostly agree with you, but ... no, you don't need that if you're talking about performance or efficiency. There's no logical or technical requirement that because the M1U has two (joined) chips, any valid comparison must also be two chips. What matters is how it performs in the real world. Heck, if that's the case, you'd need four M1 Ultras to compare to an EPYC 3rd gen, as those have eight CCDs onboard. What matters is comparing the relative performance and combined featuresets of the systems in question - i.e. their capabilities. How their makers arrived at those capabilities is an interesting part of the background for such a discussion (and a separate, interesting discussion can be had as to the reasoning and circumstances behind those choices and their pros and cons), but you can't mandate that any comparison must match some arbitrary design feature of any one system (as the 8-CCD EPYC example illustrates).You're not wrong, but I think you're taking this argument in the wrong direction. First off, this has been acknowledged and discussed earlier in this thread, in detail. Secondly, it's valid to discuss the tradeoffs and choices made by different firms designing different chips - even if I think the "and [they're] still ending up with way bigger chips too" angle from the post you quoted is a bad take in desperate need of perspective and nuance.
They also don't need that much space for actual memory , this isn't 2010 it's 2022.
Your comparison metrics are ass.
I'm not the one getting excited about this apple chip.
And as I said they still used a lot of transistors and die space to get that performance.
I,,. Wasn't doing direct comparison I'm not a deluded tool ,you can't compare disparate hardware and software ecosystems and architecture, but observational comparisons on nodes and sizes, sooo, there's that too.
That's what you're doing
"Continued intellectual dishonesty"
I don't think someone will ever be considering buying an m1 ultra or a 3970x. They are on completely different levels, the 3970x offers a level of total performance that is absolutely untouchable by the m1 ultra.
As for whether these customers are concerned about power: you're just wrong. If you're buying a hundred render boxes that will spend the vast majority of each day churning out renders? Power consumption starts mattering quite a lot, especially when you start talking about 2-300W power reductions for the same workload (assuming performance parity) (and a lot more than this if the lower power option has hardware accelerators that increase performance and lower power consumption). Assuming you're rendering 20h a day (and idling at near zero power the rest of the time, which is of course a simplification), and one render box consumes 200W with the other consuming 400W, assuming an electricity cost of US$0.112/kWh, over a year that's a savings of $163/year per render box, not accounting for the lower costs for ventilation/AC that come with dumping less heat into the room, which are also notable in warm locations. For a single box, that isn't a lot, but if you have ten, twenty, a hundred? It starts adding up. And no, this isn't an edge case - there is a relatively sizeable market for Mac-based server hardware, mostly using Mac Minis, and these will undoubtedly be migrating to Mac Studios to some extent. This is clearly not a viable competitor to the overall datacenter market, nor is it a major threat to the workstation/render farm market as a whole, but given Apple's grasp on certain parts of the software/hardware ecosystem, it is still notable. And for these people, reductions in power usage means either more money to spend on hardware for more output, or operating cost reductions. Also, an increasing amount of people and companies just genuinely want to cut their emissions. That buying new hardware is inherently contrary to that is obviously a relevant objection, but they will always need to do so at some point, and at that point it does matter which new hardware consumes the least power, as long as the difference is notable.
Also, don't discount the value of UX in hardware purchasing decisions. If you're doing CAD work or similar consistently demanding work and can't offload the heavy parts to a render box somewhere, the difference between a 600+W Threadripper Pro/Xeon+1-2-3GPU workstation on the floor next to you and this sitting on your desk? That's pretty significant in terms of user comfort, even if pro workstations also tend to be pretty well designed thermally and thus not server loud.
I definitely agree that Apple is mainly greenwashing with their talk of sustainability - but at the same time, this is also genuinely important so as a major actor, their actions actually matter somewhat even if they're hollow and shallow in many ways. Upgradeability and ease of maintenance would be fantastic, but for the market they're targeting here, that essentially doesn't happen - professionals don't have the time for downtime and troubleshooting surrounding a hardware upgrade. They buy hardware with an on-site service plan to ensure it stays working, then buy an upgrade after X years and sell/trash (hopefully the former) the old hardware. Upgradeability matters a lot more in lower end market segments, simply because those don't come with the disincentive of workin towards 24/7 uptime. But then Apple has managed the really weird feat of making regular users desire used workstation hardware, which to a large extent ensures that their old hardware gets re-sold and re-used for a long time. This is really weird that it actually works, but in this way, Apple's cult-like following actually benefits their environmental impact. I would still want them to provide in-house and third-party repair and upgrade services for at least a decade for all products - but we'll need legislation for that to happen, as Apple (as with all tech corporations) makes more money by selling you a new device. Soldered-down storage is essentially unforgiveable IMO - there's no valid reason for it - but CPU and RAM? It would be fine if there was access to replacement parts and repair/upgrade services, given the efficiency and performance benefits.
So I am curious that the low overhead on a console OS drastically improves performance over gaming on Windows, is there a similar performance gap going from Windows to Mac OS?
After watching that teardown video linked earlier, I'm actually more positive than I thought I would be in terms of repairability and upgradeability for the Mac Studio. Just the fact that every single port is modular and relatively easily replaceable? That is huge for repairability and the overall longevity and usability of a product like this. Dead ports will kill your PC far more easily than its components becoming outdated, so making them fully modular is brilliant. PC OEMs, please take note. This ought to be universally adopted.
This also just puts more weight behind my desire for Apple to implement a proper repair and upgrade service, where the motherboard is replaced. Heck, that thing has PCB area and on-board components akin to a GPU, so the embodied energy outside of the SoC itself isn't much at all. The internal design means that replacing the motherboard is essentially just replacing the SoC, as everything else is modular and can be re-used (though there are of course some onboard controllers, like 10GbE, but most of this is also integrated into the SoC). The SoC comes attached to a rather big PCB, but ports, PSU, and everything else can be kept. If they just allowed for an upgrade and trade-in programme for this, that would be a pretty decent solution for keeping these in service (of course including repair, refurbishing and re-use/re-sale of traded in parts). I'm also happy to see the storage isn't soldered down! The proprietary interface sucks, but anything else would frankly have been shocking. I didn't know they had started selling SSD upgrade kits for the Mac Pro, but having the same type of solution here speaks to at least some upgradeability (though it will no doubt cost you a ridiculous amount of money). But at least the flash modules are modular and can be upgraded. Hopefully Apple also has some sort of backdoor into their "secure" bond between flash and T2 chip so that data can be rescued off drives with dead motherboards, of course.
Also, I'm a bit surprised to see that the M1 Ultra package is the size it is - is it just me, or is that smaller than a Threadripper package? It looks a tad wider than the short side of a TR, but is definitely shorter than its long side.