Hardware reviews: Can we increase the focus on required software?

[Mussels]

Reading one of TPU's reviews and seeing the reviewer go into more detail than most websites do, still had me notice a lot of important smaller details missing - namely about issues that had plagued me over the years, and how reviews totally gloss over or miss the details on software. Screenshots dont tell users much about performance loss, slow boot times, or if quitting the software is going to cripple the hardware.

I'd raise the controversial point that motherboards should be reviewed with all their software installed, and their performance judged with all their bloatware running. I get the feeling things would shape up insanely fast.

This can become a very very deep rabbit hole, So i've stuck with examples i've ran into first hand and some external links for clarity.

The first question would be: What if a user runs linux, or has a compatibility issue with the software? What functions are lost?
The second is: Does the software come at a performance cost?

Gamers nexus had a great video on this back in 2019, where a PC running MSI dragon center and iCue at the same time suffered a massive 10% performance hit in synthetic testing, and adding an extra 30 seconds to boot times on a high end Z390, 9600K system. The gaming results... are a lot worse. A LOT worse.
According to TPU's 12900KS review, the performance difference from an i7 6700k to a 12900KS in 4K gaming is only 8%

The software to control your AIO or RGB fans can cause a greater difference than exists from a 6700k to a 12700KS.

Nothing quite like your 0.1% low FPS going from 74FPS to 34FPS to brighten up the experience your new hardware brings.

Even if you ran just NZXT cam or just iCue to control your new AIO and ignored the motherboard software, you're getting a good 10FPS reduction there... and this isn't covered in cooler reviews?
What's the use cooling it, and overclocking it to have the required software take away more performance than you started with?

What functions are hardware, what are software only?
Now some personal examples i've ran into, just to show why certain things should be covered - but they're less dramatic so they're covered by spoiler tags.

Spoiler: Logitech mouse example

I have a Logitech Triathlon M720 USB + bluetooth mouse designed to hot-swap between three devices at once.
There is a hidden button under the thumb rest, that defaults to an alt-tab function. See the little bump at the bottom to rest your hand on? Don't press in! you'll alt tab!
They actually dont even recognise this button exists officially, with the video for the mouse calling it a hidden back and forward button (Those are... quite visible. above it.)

Logitech support say to install their software, and you can disable or rebind the button! Yay! (actually they told me to RMA the mouse, which would only prevent me returning for a refund)
Once the software is installed it actually changes the button entirely in windows, so that if you press it and drag up/down/left/right, it does macro actions. This still isn't what they advertised it to be, but that parts not relevant today.
Of course, that also doesn't work on Bluetooth or alternative operating systems.

Because any changes made in the software doesn't save on the mouse, those changes do not work on the bluetooth connection methods or other operating systems (I got a K400plus keyboard and this for use with my Android smart TV, and my S22 ultra in Samsung Dex. That button screws with both, hard)

This mouse is crippled outside of a windows environment, and a review should cover that.
It also requires a crapton of sofware for its full features: Logitech Flow, Logitech Options, a firmware update tool, and the unifying receiver control tool.

What happens when the software is disconnected or crashes? Could an AIO or fans turn off and crash or damage a system?

Spoiler: Corsair Fan control issue

As a bonus point i've had a lot of software with fan controls freeze or glitch over the years (icue 3.x + HWinfo64 freezing the corsair commander pro comes to mind) where fan speeds would freeze at their current speed and stay there, even persisting past reboots.
This obviously worked fantastic when some of the default control options had 0RPM modes, spinning your fans down at idle... and they'd never spin back up again, until you restarted the corsair service manually (They actually added a button in the icue software to do this, it became such a common issue)

Future updates fixed this so it saves the settings to the fan controller at a hardware leve, making sure your fans spin up and your system wont overheat due to a silly software bug if it crashes

Corsair also offer this for their lighting controllers, and deserves praise in reviews: your lights stay how you want in the BIOS, during bootup, in sleep-mode, linux, etc

Icue 3.x even had bugs where their code for hardware monitoring was many MANY years outdated, and when used on zen CPU's and intel CPU's past a certain generation (10th?) it would prevent the CPU cores from idling, which meant it greatly reduced boost clocks on those some CPU's. Of course, having iCue poll your CPU temperatures was extremely important if you installed it for your headset and didnt have any corsair cooling products. So, would you take tanked CPU performance or a headset without any of its primary features?

If you have an AIO with USB Control, you want to know what profile it's running without the software.
If you get a USB controlled fan hub, you want to know if the fans will get stuck at 0% while windows updates are installing before the software has loaded.




What I'd like to see in the reviews:

What features are software dependent?:
Is the software required and what features won't work without it?
What's the performance impact of running the software? (CPU benchmarks, brief game tests focused on CPU sensitive titles, RAM usage, boot times)
Can you save hardware profiles to the device directly? (Fan speeds, pump speeds, ARGB lighting, mouse/keyboard rebinds and macros, etc)
Does it forcibly bundle extra software, or try to trick the user into installing extras? (Gigabyte are guilty of both of these)

Performance impacts of software:
What is the performance cost of running the software? (Before/after CPU testing should be simple enough)
Do you get worse battery life/power consumption with or without the software? (Forced default RGB lighting on wireless devices, etc)





Performance results comparing:
*Clean OS
*Default install of mandatory software
*Features enabled (ARGB lighting, fan control) - Is firing up RGB fusion to control our systems lighting going to chew up an entire CPU core? (Yeah, it's happened with other products/brands)









****** Below is the original post that i was going to put on @ir_cow's review of the Z690 waterforce motherboard review thread, and moved here instead. It covers what i was thinking of asking, and why*****

What happens if i don't install the software? What features do I lose? Does the OLED display work without software and drivers? (Linux users can't install the gigabyte software)
Since the board requires the Aorus app center, what else is bundled with that software? (1)
Does it install and activate any always on services or programs? (2) (This ones tricky, since a lot of this stuff is not immediately obvious)
What stops functioning entirely if the software isn't running? (3)
What happens to RGB devices without the software running? (4)
Are there alternative software options? (5) (review does briefly mention this, but not by name or what OS they work in)
Does the software mesh well with other officially supported brands? (6) (If you're reviewing a motherboard, you'd test how it controls RGB ram lighting. Reviewing the RAM? Test it on various brands mobo software)
Does X feature have a performance impact? (If you're testing a motherboard with ARGB lighting, you absolutely need to connect ARGB fans and strips to that motherboard, fire up the software and see if CPU usage goes sky high at idle, because of a rainbow animation. corsair had this issue with 'link lighting' syncing devices)

Can a linux user still use the OLED display of that motherboard? Will it give temperature readings without any software at all, and how accurate are they? (Or, what sensor does it read that matches up with the BIOS/HWinfo64 etc)
Does the board remember or forget previously set settings from the software? (Ex: reboot to install windows and a fan controller freezes, locking someones AIO at 0%)



A few youtubers have gone down that road in the past, but written reviews always cover more ground and after a few nightmares with software i tend to avoid hardware that requires software to run more than anything else these days




(1) My gigabyte monitor software installs Gskill trident Z RAM RGB lighting software. I have corsair ram. There is no option to NOT install it.
(2) Asus motherboards push to auto-install software that does this, running an always on service that remains even after the software is uninstalled. This service causes BSOD's if you move that SSD to another non-asus system, and would get totally missed by reviewers who clean install for every test but also screws with a lot of home users doing hardware upgrades who never knew about it.
(3) Corsair did shenanigans requiring users to use iCue 4.0 without allowing all devices to use it, which rendered one of my AIO's to run in it's default hardware state. All pump control and fan control was lost. A Corsair headset i own also got crippled, losing all positional sound, RGB lighting control and power saving modes.
(3a)Corsair also has a win here, where many of their RGB products (AIO's and headsets often excluded) have 'hardware lighting' profiles that allow you to run the software once, save settings to the device and never need the software installed again
(4) My Gigabyte AX370 Gaming 5 has full RGB control in the BIOS itself, and my Asus x570-F allows you to bind the reset button of your case to limited RGB control.
(5) Corsair RGB ram can only be controlled by iCue, while more generic RAM might be linked and controlled by half a dozen motherboard vendors softwares.
(6) I bought LiFX LED lights for my walls because they advertised support for razer synapse. Not one review mentioned this requires an app that doesnt even support minimizing to the tray, it must be always open and always on-screen for that to work.

 

[W1zzard]

I never install any software for GPU or SSD reviews. For CPUs I install the chipset driver. Never install any motherboard software

 

[Vayra86]

Dunno, but I always, and I think we always tell people to troubleshoot their frametime/fps woes by returning to stock.

Its common sense. Run lighter, means run faster. The fact is that commerce (and as a result: popular opinion) clouds the view of common sense. Maybe we should review our own psychology

I keep telling people around me much of the same idea: the more crap you run in and around the house, the more you'll worry about keeping it all going. When you own a house, you start noticing. Every bit of comfort brings a bit of maintenance. And what if you can't fully do that maintenance the exact way you want it? Then you've bought into something that causes more trouble than it solves. The perfect commercial story: create a need where there wasn't one. Note: every single smart home appliance is firmly in this category. And those are remarkably similar to PC accessories like RGB or numerous feature buttons that go beyond keyboard functions.

RGB and all that other additional crap we use in and around our PCs is exactly the same. If it doesn't cover bare needs, you don't need it and it probably won't help you. And if you can't fully control it and change it yourself, you've been enslaved to it - while paying to get there.

So rather than making reviews even more complicated, how about getting that general idea ingrained in everything we do? Otherwise where do you stop?

 

[chrcoluk]

I remember many years ago I started using asus OC software and it used something ridicolous like 20% cpu at idle, I uninstalled it sharpish.

 

[BSim500]

The problem with this stuff is the sheer number of permutations, ie, if software A, B, C, and D reduces performance, user X has A, B & D installed, user Y has B & C, user Z has A, C and D, etc, that's going to get messy trying to benchmark that to be useful for everyone. For benchmarking, I'd say keep the configuration clean and let the user worry about whatever bloated RGB software they insist on installing (I'm sure there's more efficient software (see below) out there if they truly wanted it). Rather than review every single motherboard with the software installed (most of us who are sane have long learned to avoid that), you could do a software review and the before /after performance impact it has once, then link to it in reviews of board that bundle that software otherwise you'll just ending up duplicating the workload across every single board of the same brand.

Same with Logitech Gaming Mouse software, best feature I love about my G300S mouse = you bind the 6x extra mouse buttons as you want them once and the info is then permanently stored in flash inside the mouse. So if you permanently bind the keys to usually unused keys like /*-+0. numpad keys, then just match the same keys in-game, eg, if you bind mouse buttons 4 & 5 to / and * (Numpad keys), then in-games you just bind the same / * keys to say Inventory and Map, no Logitech software is needed to be constantly installed for that. Plus it works with other OS's, eg, Linux. And it feels awesome being able to dedicate the same mouse buttons to that across hundreds of games without any background bloat.

Edit: And for "Gigabyte monitor", surely it's better to use general brand-neutral monitor software (eg, CPU-z, GPU-z, HWMonitor, HWInfo, Coretemp, CrystalDiskInfo, MSI Afterburner, OpenRGB, etc)? I haven't used nor seen the need for brand-specific monitors in years.

 

[R-T-B]

gigabyte would become snail if we did this.

 

[pavle]

I never install any software for GPU or SSD reviews. For CPUs I install the chipset driver. Never install any motherboard software

Indeed. Experience through the years shows that most software accompanying hardware and even drivers were poorly written so I learned to always install only drivers, then find software that is best suited for my interests.

 

[R0H1T]

I only install anything extra to install stuff like firmware, especially for SSD's, everything else I get from MS updates or the manufacturer's own site! And screw that RGB software, it's by far the worst trend in PC landscape this past decade

In fact I also disable useless Windows services after a new install.

 

[Deleted member 24505]

I don't use any software for RGB control that runs in the background, just OpenRGB and all my fans are directly connected to MB RGB headers. Imo all the RGB control software iCue etc are all bloaty shite.

 

[qubit]

I do like the idea of getting our PCs to huff and puff with overclocking just to get back the performance lost by the software - and with less stability. So gloriously pointless!

 

[ir_cow]

@Mussels I know you mean well, but as a TPU staff member there is other ways to express your wants & desires less publicly. We have staff forums, private messaging, emails and other methods for communication.....

I will take your feedback under consideration for future reviews. Just understand some things are just not worth covering or feasible. For example, checking if the RGB software works (for RAM) on every MB vendor seems like a straight forward thing, but really that requires having Gigabyte, MSI, ASRock, ASUS, Biostar, EVGA, on hand. Each either having a OS SSD dedicated to each or the software uninstalled/reinstall each time. Afterwards checking if things like Static, Pulse, Flash, RGB Cycling, Music, Wave, etc all work and reporting each brand. I'll tell you right now, it does not all work. You can only count on Static and RGB cycling to to work unless you install that memory's specific software. If something doesn't work correctly, go to Gigabyte, MSI, ASRock, ASUS, Biostar, EVGA and submit a request for it to be fixed.

I don't think its worth the time to investigate when a week later a update comes out and fixes whatever was the issue. Suddenly that 10+ hours of investigation has be void and nulled. Either the article needs to be amended or the section removed completely. As a reviewer I do not have time to go back to check every bit of software again just to see if something changed.

On that note, if some windows software is required to run, it will be in the review. In this case you assumed wrong. AORUS app center isn't required for the OLED to work, leak sensor or temps. Only reason to install Gigabyte apps is if you want to use the RGB software instead of OpenRGB, or want to use the CPU OC tools instead of setting it up in the BIOS.

 

[Mussels]

@Mussels I know you mean well, but as a TPU staff member there is other ways to express your wants & desires less publicly. We have staff forums, private messaging, emails and other methods for communication.....

I will take your feedback under consideration for future reviews. Just understand some things are just not worth covering or feasible. For example, checking if the RGB software works (for RAM) on every MB vendor seems like a straight forward thing, but really that requires having Gigabyte, MSI, ASRock, ASUS, Biostar, EVGA, on hand. Each either having a OS SSD dedicated to each or the software uninstalled/reinstall each time. Afterwards checking if things like Static, Pulse, Flash, RGB Cycling, Music, Wave, etc all work and reporting each brand. I'll tell you right now, it does not all work. You can only count on Static and RGB cycling to to work unless you install that memory's specific software. If something doesn't work correctly, go to Gigabyte, MSI, ASRock, ASUS, Biostar, EVGA and submit a request for it to be fixed.

I don't think its worth the time to investigate when a week later a update comes out and fixes whatever was the issue. Suddenly that 10+ hours of investigation has be void and nulled. Either the article needs to be amended or the section removed completely. As a reviewer I do not have time to go back to check every bit of software again just to see if something changed.

On that note, if some windows software is required to run, it will be in the review. In this case you assumed wrong. AORUS app center isn't required for the OLED to work, leak sensor or temps. Only reason to install Gigabyte apps is if you want to use the RGB software instead of OpenRGB, or want to use the CPU OC tools instead of setting it up in the BIOS.

The feedback forum is exactly for this - bringing in ideas, and getting feedback from everyone.

I meant no insult to you or your review if you interpreted it that way - it was just a thought that occurred while reading the review that got too big to belong in the reviews discussion thread

 

[Ferrum Master]

Well the thing here is about the obvious. If you had the pleasure setting up PC's for DAW with capricious PA equipment, then you would eat your hat on each crap driver doing stupid IRQs. I never use that software bloat. openRGB for RGB and all other settings are exclusively set in BIOS it will be like that. I mutliboot Linux and none of those would work there anyways. I can't remember when we first had those things... around 2000... if also take into account some early anomalies like starforce that pushed even me to pirate the game just because.

The UEFI BIOS payload triggering to install this crap should be also banned altogether, yet there are claims that there is not enough space for AGESA. Some have made UEFI BIOS and ugly mess too and that needs treatment too.

I don't understand the thing being touchy about Mussels expressing his concern here, that's not professional.

There is some sort of Asian culture touch about these softwares... they are like force to be there without thinking how the RoW would react on them, especially we like minimalist approach. More over, they exist just because of the marketing and keeping the crapware team alive.

 

[Vayra86]

I don't use any software for RGB control that runs in the background, just OpenRGB and all my fans are directly connected to MB RGB headers. Imo all the RGB control software iCue etc are all bloaty shite.

Precisely. You simply know, or should know by now that all bloat is in fact bloat. A review could easily point that out by placing it under Cons every single time: 'Comes with bloatware'. If there is a better (open source/free) application to do a task, a company spent resources to build something shittier that we should avoid.

That's what reviews are for and it is how they can influence the industry and customer base, by pointing out inside knowledge of those with experience.

@Mussels the problem with 'measuring' how bad bloatware really is, is a justification of its existence. "Oh, but its only 4%" is the next cognitive dissonance move for buyers. And we know by now that's how people work, otherwise buyer's remorse pops up and ooh that doesn't feel nice. Additionally, 'CPU Load' or resources in use don't translate to similar performance issues, like someone else said, there are so many variables in play here, what do you really measure?

 

[ir_cow]

Its only bloatware if you have to use it and causes performance drains.

Vaild points are being made here and should be discussed in the review. Worth a article at some point installing all the software and seeing what the dropoff is. I think the days of 1-4 cpus and 4GB of ram has convinced people all optional software is bloatware.

Isn't Intel Atom (E-cores) exactly for this? Taking care of small background tasks.

 

[ThrashZone]

Hi,
Even mother board manufactures are hard pushing their crapware abilities to get better performance than manually doing it yourself
So at some point crapware has to be tested to.

Not sure I've seen this added to reviews setup info either it's not just for gpu either

Windows 11 Tweaks for GPU Benchmark

Updated for 24H2 Install without Internet When it prompts you to go online, press Shift+F10 and type "OOBE\BYPASSNRO" (that's an o not a zero at the end). After the automatic reboot you can install without network Install on systems without TPM, UEFI or other requirements...

www.techpowerup.com

Windows 10 Tweaks for VGA Benchmark

Updated for Windows 10 21H2 Update - Install without Internet, so you can create an offline user - name the user "TPU" (the user will be deleted at the end of the scripts and you're using "Administrator") - Install VGA driver, from USB, still without network, so Windows Update won't install a...

www.techpowerup.com

 

[thewan]

You are too narrow minded. you should get out of your shell (pun intended) and broaden your horizons. You say logitech devices are locked outside windows? For starters, Logitech Options+ is available for Mac. That already makes your statement a lie.
Second, linux users for the most part, are savvy enough to find 3rd party applications or drivers for their own use cases, especially for big name brands like logitech. The wonderful thing about the open source community is how they contribute to make all these hardware function the same or nearly the same as it does on their Windows counterparts.
Third, you mention android. Same thing applies, download a button remapping app, and voila, modify ANY hardware key/button, be it on your phone or external wired/wireless input device, you want to. A simple google tells me this can be done.
Maybe if you think a little bit outside the box you would at least google before posting the above.

 

[Deleted member 24505]

Its only bloatware if you have to use it and causes performance drains.

Vaild points are being made here and should be discussed in the review. Worth a article at some point installing all the software and seeing what the dropoff is. I think the days of 1-4 cpus and 4GB of ram has convinced people all optional software is bloatware.

Isn't Intel Atom (E-cores) exactly for this? Taking care of small background tasks.

The E cores are Gracemont not Atom

 

[Deleted member 202104]

The E cores are Gracemont not Atom

From Wikipedia

Gracemont is the 4th generation out-of-order low-power Atom microarchitecture, built on the Intel 7 manufacturing process.

Gracemont (microarchitecture) - Wikipedia

en.wikipedia.org

 

[Deleted member 24505]

The hybrid design featured 2 P-Cores (Golden Cove) and 8 E-Cores (Gracemont).

https://www.google.com/amp/s/wccfte...ers-50-higher-multi-threaded-performance/amp/

4th gen Atom or not, still Gracemont

 

[Assimilator]

No.

TPU's hardware reviewers don't test systems, they test components. And they generally do a very thorough job. Forcing them to introduce additional software, AKA additional variables, into their testing for... no good reason I can see, will mean they either test less hardware, or they do it less thoroughly. Neither of those is worth losing.

Yes it would be ideal for reviewers to test the "whole package", but ultimately the onus needs to be on users to do their research and not pull the trigger on a purchase after reading one review. Certain brands are known shit shows in terms of software and evidence for that can easily be found via search engines.

What if a user runs linux?

Hate to break this to you, but you're the only one who gives a shit about Linux on the desktop. The best chance for "desktop Linux" to be a thing was a decade ago, then Macs got the ability to play videogames and that was it.

Yes, I know Valve is trying to make Proton a thing. But the only reason they're doing that is to make the Steam Deck a thing and lock more people into the Steam ecosystem, not because they give a shit about Linux.

 

[80-watt Hamster]

Hate to break this to you, but you're the only one who gives a shit about Linux on the desktop.

Not true at all. Not personally running it right now, but am seriously considering giving desktop Linux another go. I had my first go-round with Linux when consumer Windows was Win32 and sucked. Then Microsoft got consumer Windows on the NT branch, and life with Windows was much better, better enough to make multiple systems and/or dual booting more trouble than it was worth (for me). So Linux and I broke up. Now Windows is starting to suck again. Getting back together with Linux is going to largely depend on what Windows is like when W10 reaches EoS.

 

[johnspack]

Do give Ubuntu a try Hamster. You can install wine, dxvk and vkd3d-proton yourself, and run most windows games ect without steam even.
If you want to dual boot, use separate drives and use bios to select the boot media. Win 10 kind of wrecked dual booting.
I do have win 10 21h2 installed as well, but rarely ever boot it. I ran windows from 3.0 to 8.1... then gave up at 10. Linux does everything I need.

 

[Assimilator]

Not true at all. Not personally running it right now, but am seriously considering giving desktop Linux another go. I had my first go-round with Linux when consumer Windows was Win32 and sucked. Then Microsoft got consumer Windows on the NT branch, and life with Windows was much better, better enough to make multiple systems and/or dual booting more trouble than it was worth (for me). So Linux and I broke up. Now Windows is starting to suck again. Getting back together with Linux is going to largely depend on what Windows is like when W10 reaches EoS.

Cool, so there's an entire two of you.

 

[80-watt Hamster]

Cool, so there's an entire two of you.

Do you work at being this cynical?