Thursday, March 17th 2022
Apple's Graphics Performance Claims Proven Exaggerated by Mac Studio Reviews
Apple made some bold claims at the launch of its new Mac Studio computers when it came to the performance of the new systems and it looks like Apple was exaggerating those claims by quite some margin when it comes to the graphics performance. The first reviews of the new Mac Studio went live today and thanks to those reviews, despite the limited benchmarks that were performed on the new systems from Apple, that as so often Apple's performance metrics are still relying on the reality distortion field. Most of the publications that got their hands on the new systems focused on CPU benchmarks and there's no doubt the Ultra version of the M1 processor is a beast when used for things like video rendering and complex image manipulation, where it's butting heads with an AMD Ryzen Threadripper 3970X.
However, Apple's 64-core GPU isn't quite what the company claimed. In the presentation footnotes Apple provided details on the "highest-end discrete GPU" that they compared to, which was an NVIDIA GeForce RTX 3090. If we were to be kind to Apple, we would say that the company was slightly off target here, but it's actually not even remotely close. Tom's Guide tested the M1 Ultra SoC in Sid Meier's Civilization 6 and got a whopping 38.85 FPS at 1440p, which is beaten soundly by a Lenovo ThinkPad P1 Gen 4 with a GeForce RTX 3070 laptop GPU that scored 64.9 FPS at 4K. Likewise, The Verge decided to test the claims and had a system with an actual GeForce RTX 3090 in it and ran Shadow of the Tomb Raider and the PC managed 142 FPS at 1080p, with the M1 Ultra coming in quite far behind at 108 FPS. Moving up to 1440p the 3090 came in at 114 FPS, with the M1 still trailing behind, if not quite as badly at 96 FPS.The issue here isn't the actual performance, as it's no doubt decent figures considering the max power draw of the Mac Studio is 370 W, far less than any PC with a GeForce RTX 3090 in it. However, Apple's inflated performance claims are getting old and tired and the company really needs to do better. It's highly unlikely that anyone will be buying a US$4999 Mac Studio to play games on, as that's what the base model with the 64-core GPU costs. As such, Apple should provide realistic benchmark comparisons, as overall the Mac Studio is an impressive piece of kit in its own right. Based on the first reviews, it's likely going to end up being used by a lot of creative companies that would normally use Macs for their workflow.
Sources:
Tom's Guide, The Verge
However, Apple's 64-core GPU isn't quite what the company claimed. In the presentation footnotes Apple provided details on the "highest-end discrete GPU" that they compared to, which was an NVIDIA GeForce RTX 3090. If we were to be kind to Apple, we would say that the company was slightly off target here, but it's actually not even remotely close. Tom's Guide tested the M1 Ultra SoC in Sid Meier's Civilization 6 and got a whopping 38.85 FPS at 1440p, which is beaten soundly by a Lenovo ThinkPad P1 Gen 4 with a GeForce RTX 3070 laptop GPU that scored 64.9 FPS at 4K. Likewise, The Verge decided to test the claims and had a system with an actual GeForce RTX 3090 in it and ran Shadow of the Tomb Raider and the PC managed 142 FPS at 1080p, with the M1 Ultra coming in quite far behind at 108 FPS. Moving up to 1440p the 3090 came in at 114 FPS, with the M1 still trailing behind, if not quite as badly at 96 FPS.The issue here isn't the actual performance, as it's no doubt decent figures considering the max power draw of the Mac Studio is 370 W, far less than any PC with a GeForce RTX 3090 in it. However, Apple's inflated performance claims are getting old and tired and the company really needs to do better. It's highly unlikely that anyone will be buying a US$4999 Mac Studio to play games on, as that's what the base model with the 64-core GPU costs. As such, Apple should provide realistic benchmark comparisons, as overall the Mac Studio is an impressive piece of kit in its own right. Based on the first reviews, it's likely going to end up being used by a lot of creative companies that would normally use Macs for their workflow.


151 Comments on Apple's Graphics Performance Claims Proven Exaggerated by Mac Studio Reviews
I care that Apple said it's faster than a 3090, when it absolutely is not in any performance comparison that anyone has been able to show.
That's not an exaggeration, nor an error by omission, nor a mistake - it's an absolutely shameless bald-faced lie. It doesn't surprise me, because that's the kind of company Apple is and the kind of morons they sell their overpriced garbage to, but this is a tech forum and I expected better from its members than apologism for such a company.
I have no problem with talking with people on the internet. I have serious problems with those who have drunk the kool-aid.
Did any of our supposed "tech" press run comparisons with the list of games I suggested? The native games? Did anyone here look at industry benchmarks against 300W 3090s? No. They are not interested. They want to pass some narrative that the M1 Ultra is slow, for some reason. Meanwhile I don't even expect Intel's fastest GPU to touch the Mac Silicon. And yes the CPU in the Mac silicon slows things down in single core at only 3.2ghz also, but you can always test at 4k resolution if you want a more GPU limited result. I guess Hardware Unboxed will do the work since Apple fans don't bother, and PC users don't care about the truth anyways.
Nobody is suggesting that Apple products are reasonably priced, but the Mac Studio is a good product. That horrible abomination called the Studio Display? That's another matter.Right so you have a selection of industry benchmarks against a 300W RTX 3090 to share? Waiting.
Do you have benchmarks against a 160W 12900k? Because I do.
Cinebench 12900k at ~160W TDP: ~19k
M1 Ultra at 100W lower than that limited TDP: 24k
Apple was more than 20 percent faster CPU wise.
That's what Apple "said" through their efficiency curve graph, and they did not lie. The M1 Ultra was also dramatically faster at Firefox compiling tests and code compiling in general. And because of the media encoding engines we don't even want to compare against the RTX or Intel competition, Apple has dedicated hardware for that.
As for the GPU, I'm still waiting for a scientifically literate person to run a comparison of industry benchmarks at 300W. As I've said though, it is Apple's fault for not specifying the benchmarks and conditions clearly for people. Especially since most people can't even read a perf/watt graph properly. (x,y) co-ordinates being too much for most people.
In no way did Apple say their chip was better at gaming. As for that, I expect we'll find that the Apple GPU is 20 percent faster than an RTX 3070, and 10-15 percent slower than 300W TDP limited 3080/3090s. And faster than a 300W limited 3090 in productivity benchmarks, as Apple said.
Apple can take any piece of hardware and put power restrictions on it, then claim "Mac is faster!" In fact, that is what they are doing to make the M1 look like a "high performance" chip.
BTW, I can run faster than Usain Bolt.*
*When Usain Bolt is 75 years old, overweight, and has had one foot amputated.
The difference between my claim and Apple's is that I added a disclaimer.
I also doubt that the m1 will beat a 300w plimited 3090. I mean i have a 3090, do you want me to run something specific?
Also, just a side note, the sotr numbers are way off. My 3090 gets 196 fps at 1440p highest, yet in the verge review it hovers around 110? Hmmm
I expect Apple would have used stock voltage curves for their Intel and nVidia power consumption results, they would have just used wattage limits instead. For sure you can downvolt the 12900k manually a lot, if you get a golden sample. My 12700k is a golden sample.
I did find one reviewer to talk intelligently about what Apple was claiming, and one youtuber mentioned it, that was The Verge and MKBHD (though I still think overall Ars Technica did a better job with their review). Here's their graph pointing out the 3090 does scale to over 400W. They get it. You can be faster at 200W less power, and slower overall at unlimited wattage. We'll probably get the most benchmarks from MaxTech (they do a lot of that) and HardwareUnboxed (they are the king at that) eventually, if they get a Studio in hand one day. They'd have to buy it, so now sure if that will happen.
It's worth remembering Anandtech's review: "I’m not entirely sure what that says about Intel’s 7 manufacturing process compared to the 10SF used before. A lot of the performance gains here appear to come from IPC and DDR5, and that doesn’t seem to have come with performance per watt gains on the P-cores. It means that Intel is still losing on power efficiency at load compared to the competition."
Apple has got the future with their current products. Perf/watt is king in gaming consoles, in laptops, in mobile, in everything except the giant 500W workstations.
Cheers :)
Reviews are at the mercy of the interpreter, it can be read in various ways and if someone was.... fanboi enough?, they can twist it anyway they want. PRs should never be taken as fact without a way to back any numbers, this is why we wait for reviews. One review doesnt say a lot, the sample size is too small and if problems show up for them, they could be one off or more serious, multiple reviews become a necessity.
Using Geekbench after everyone admits its useless and still using it for factual numbers is completely stupid. Corporate PR departments dont care what they put out is the truth or an exaggeration of said truth.
I find most of the discussion here goes good with the popcorn. :D
Mac Studio FULL Teardown - M1 Ultra chip REVEALED! - YouTube
Try that youtube channel :)
Thing is you can't compare a $5000 MAC to consumer parts because well you are competing with Workstations at that point. Workstations that if something goes wrong with a single part can be replaced. In most instances down the road you can also upgrade them with more Ram, multiple nvme, ssd's or HDD's. etc. MAC's at this point are one and done. You drop $5000 and that MAC will never be upgraded. Once a single part of it isn't usable anymore or outdated the whole thing is along with it.
I also wonder how long Apple is going to be able to keep making giant leaps in its speeds without astronomical R&D costs. Apple doesn't sell to anyone but through their products. Intel and AMD keep selling to game console makers, have licensing with their GPU technology in monitors and other technologies they create. They have multiple revenue streams and licensing deals. They make server parks and workstation parts as well as consumer parts. I'm not saying Apple can't be successful but they neglected a lot of things for a long time and had turned into a cell phone manufacturer. How many companies are going to trust them to switch to their hardware and expect Apple to keep making large leaps and to guarantee that the software always runs best on MAC.
I'm not really into 'coverage' videos, those kinds of video are more hype than fact, but I still looked over that YT channel, the guy appears to be an Apple schill looking at his content history. That video is about a breakdown of the Ultra, not benchmarks. Until more reviews are made this back and forth is moot.
And with that I'll step out of this thread for fear of being called names.
Apple was not dishonest (outside of not defining their productivity bench, but the curve is not a lie, it is a focus on their efficiency). The posted perf/watt curves. Then since their own product only uses 100 watts, they moved across the graph on the axis to see how many watts a 3090 used to get the same performance, and it was 300W, and they stopped there. I think Apple overestimated the average journalist's ability to understand basic mathematical graphs. They never claimed "we are faster than a 3090". If you read that somewhere, a journalist wrote that, not Apple. I am a mathematician and frequently look at graphs like that. Pick x, follow y. Pick y, follow x. That's how you get reference points for comparisons on a curve.I did provide sources for almost everything I said. You picked one comment, and complain that I didn't provide a citation, and you're trying to use that to discredit everything I said? That's not in good faith.
They did not say "we can play games at a higher fps", if they wanted to claim that, they would have shown FPS data. 3DMark Wildlife shows the GPU is about 21 percent faster than an RTX 3070 and probably ~10 percent slower than an RTX 3090 at 300W, but of course, as The Verge said, it was not good and is very misleading, as Apple "CROPPED" the graph and didn't show where the 3090 goes with 450W for example. The 3090 is probably at least 30 percent faster than the Apple silicon, with unlocked power. But the Mac Studio has no option to unlock clock speeds or power, so of course they didn't want that comparison.
I still think Apple isn't being dishonest. If everyone reads Apple's charts and concludes the wrong things, you can't say because those claims are not true that Apple is dishonest. Apple didn't make those claims. What we should do is heavily criticize Apple for misleading marketing. But guess which companies are the most famous for misleading claims? NVidia and Intel are the worst. Won't go in to that.
Apple has problems. Mac OS is a mess. Gaming is a mess. They need a turbo option for their CPUs so we can get higher clock speeds at lower usage. And they charge too much. They make luxury products, not value for the dollar. They are also making the most exciting silicon in the business. Would that I could install Windows on the Mac Studio.
I hope software improves over time to work on two GPU dies stitched together. We'll see. Intel and AMD have big plans here. nVidia probably also. Apple did it first.
Never happened before...
But I love the Studio make no mistake about it.
Threadripper 3970X: MSRP $1999 (PCPartpicker says $3100 real-world)
Suitable motherboard: Cheapest is ASRock TRX40 Creator at $516
RAM: 4x16GB of DDR4-3600 is ~$260 at the minimum
Storage: ~$150 for 1TB of high speed PCIe 4.0 NVMe (WD SN850 or Samsung 980 Pro)
Cooler: Noctua NH-U14S TR4-SP3 $90
GPU: RTX 3070 or RX 6700 XT - $600-800 for an RX 6700 XT or RTX 3070 - subtract ~$2-300 for prices in a saner world
PSU: >650W 80+ Platinum or Titanium with a good design, ~$130
Case: Whatever you want, but ideally a well ventilated one, likely >$100
Windows licence: $100+
That leaves us with a total price of ~$3950-$4200 depending mostly on GPU choice (or ~$3750-3900 if GPU prices were less stupid), with performance very much in the same ball park, power consumption easily being 3x under heavy loads, a case many times the size, but far more upgradeability and expandability.
So, will this spec wise "kill the Mac"? No. Is it a tad more powerful? Sure - the TR 3970 is a bit faster in many workloads, and the GPUs used in the example are a bit faster than the 48-core M1 GPU. It's still quite comparable overall. You can beat it by going for a higher end CPU or GPU, but that will also (drastically) increase your price - another $2000 for a TR 3990X, another $1000 for an RTX 3090 or RX 6900 XT. RAM and storage upgrades are of course much cheaper, and the advantage from expandability shouldn't be undersold - it's important both for utility and longevity. But neither should the M1 advantage from its NPU (RTX cards can do some similar work with tensor cores, but far less efficiently, and otherwise the PC has no neural accelerators at all unless you add an accelerator card for even more cost), nor its massive potential for VRAM - if you have workloads that can make use of this. You might of course not - but that doesn't invalidate those examples where the advantage applies.
So, is the M1 Ultra better or worse than a HEDT PC? It's mainly different. It's still massively powerful, but not the most powerful; it's massively efficient, but not great at all workloads; it's extremely fast for certain specific workloads, but those are quite specific.When you're comparing efficiency curves it's a reasonably valid comparison. If you're comparing your part, which peaks at X power, it's reasonable to compare that to competitors also at X power, even if they scale much higher than that. It's not the full picture, and a fair comparison would also include competitor performance at Y (their peak) power, but that doesn't invalidate the X-X comparison or make it dishonest - it's just selective. Selective argumentation can indeed be misleading and can hide the truth, but it's quite a stretch to call this example lying or outright dishonest.
Also, no 3090 at a 3070's power level would be slower - a wide-and-slow design is nearly always much more efficient than a smaller, higher clocked one. That's a huge part of why Apple is managing their massive efficiency - they're just making huge cores and huge chips and running them at relatively modest clocks. But unless specifically bottlenecked elsewhere, and unless you go really unreasonably low in power, a low-clocked large GPU will always be faster than a smaller GPU at the same power levels. There's a reason Nvidia's mobile SKUs nearly always have more CUDA cores but lower clocks than their desktop counterparts - that's how you improve efficiency in a given thermal envelope. It's just expensive to do so.
Here is mine at 35w beating the m1 at both efficiency and performance
About the 3090 vs 3070, I'm not sure its so clear cut. It always depends on the workload and whether or not they can feed those cores. Im pretty sure that a power limited 3090 at 720p gaming will perform worse than a 3070. .Im sure the 3970x is over twice as fast as the m1 ultra. Going by cbr23 results, it score 47+k. So what do you mean same ballpark?
And no, a 3090 power limited to the same level as a 3070 will not in general perform worse at 720p. It might in some games - certain applications don't scale equally well in parallel, or hit execution ceilings that need higher clocks or IPC to be overcome - but in general, the 3090 will always be faster in GPU-bound workloads. It could theoretically be VRAM limited, but with 24GB that is quite unlikely, and no games come close to saturating a PCIe 4.0 x16 connection, so that isn't a differentiating factor either. And given how voltage/frequency curves work - the power cost for increasing clocks increases as you push clocks higher - a lower clocked, wider GPU will almost always be more efficient than a smaller, higher clocked GPU at the same power levels.Cinebench seems to be somewhat of an outlier in terms of M1 performance (which might be down to how well it scales with SMT - that's a big part of AMD's historical advantage over Intel in that comparison, at least, as their SMT is better than Intel's). Not that GeekBench is a representative benchmark either, but in that, the M1U scores ~24000compared to 26500 for a 3975WX system. That 3975WX system is running quite slow RAM (DDR4-2400) but with 2x the channels of a 3970X it should still overall be faster in anything affected by memory bandwidth. Still, that's a ballpark result IMO, even if it is still a clear victory for the TR chip. "A bit faster", like I said above, fits pretty well.