Desktop System Review Overhaul

[crazyeyesreaper]

Recently I reduced the number of tested systems shown in Desktop reviews in order to make the graphs cleaner and easier to read while providing charts detailing their specifications with less overlap between mini-PCs, laptops, and desktops. However I pose the following question to the community.

What coverage of systems do you consider preferable?

Keep in mind in many situations I cannot typically keep or re-bench systems later this means continued comparisons become more cumbersome as time goes on unless I get lucky and a system is lent out long term. As such a comparison between multiple systems with the option of re-benching later or adding new tests becomes increasingly cumbersome.

The other option is to focus on the system at hand while I have it and run more tests giving deeper coverage of a reviewed system. This means benchmarks can be updated or swapped out far more easily. Example dropping the venerable Battlefield 4 for something else.

Let me know your thoughts below.

 

[the54thvoid]

It's more complex to compare systems to one another, I'd agree there. What you can easily do is reference running clocks on spec, for example, informing the reader if the core clock of a GPU in the build (or CPU for that matter) is lower than brand 'x' for thermal reasons. Other than that, aesthetics, acoustics, etc can all be gauged by the author (you) and imparted to the reader. What could be amusing, but would involve a little extra work, is the DIY comparison cost of such a review, i.e. 'can you build it better/cheaper yourself?'.

 

[crazyeyesreaper]

It's more complex to compare systems to one another, I'd agree there. What you can easily do is reference running clocks on spec, for example, informing the reader if the core clock of a GPU in the build (or CPU for that matter) is lower than brand 'x' for thermal reasons. Other than that, aesthetics, acoustics, etc can all be gauged by the author (you) and imparted to the reader. What could be amusing, but would involve a little extra work, is the DIY comparison cost of such a review, i.e. 'can you build it better/cheaper yourself?'.

I typically touch on build cost in the Desktop reviews while I do not detail a similar spec system with less features I do tend to see if a similar system with the same parts as close as possible can be built and for what cost. I also already look at and touch on temps, and clock speed differences etc. Along with worst case scenario thermals by using excessive testing to see if the system will encounter thermal throttling or other oddies (example Dell laptop review it needed a new bios for Intel's turbo function to even work properly as dell left the option out of the bios)

Basically it breaks down as i can compare systems but that means game tests / benchmarks remain the same and can become stagnant. Breaking free from that and focusing on the singular systems means as new games come out i can more add or remove tests (while still keeping a few long term options in place for those in need of a comparison)

 

[eidairaman1]

I'd like to see gpu tests redone after maybe 2-4 years, just to see how older hardware stacks up to newer.

 

[crazyeyesreaper]

I'd like to see gpu tests redone after maybe 2-4 years, just to see how older hardware stacks up to newer.

That would mean id have to be able to keep the systems... which 99% of the time is not the case.

 

[rtwjunkie]

Yours is a different case from W1zzard, in which he keeps GPU’s (the reference models usually) for a long while.

Someone interested in a complete system is normally predisposed to a system, and wants more information on it to answe all their questions.

Therefore my vote is just to go more in depth on the review of that one system. People can also read other reviews of other systems and get a fairly good comparison, even if some factors of testing are different.

 

[lexluthermiester]

Yours is a different case from W1zzard, in which he keeps GPU’s (the reference models usually) for a long while.

Someone interested in a complete system is normally predisposed to a system, and wants more information on it to answe all their questions.

Therefore my vote is just to go more in depth on the review of that one system. People can also read other reviews of other systems and get a fairly good comparison, even if some factors of testing are different.

That was my vote and for the same reason. Individual benchmark results will offer a clear enough picture to prospective users to make an informed choice. System comparisons, while useful, are not practical in all situations such as crazy's.

 

[bonehead123]

^^what lex said^^

I too would prefer a focus on the current system's performance, features, costs, and notable downsides, if any....

IMO, comparing 2 (or 3) systems to each other is only valid if they all contain current-generation components with similar specs and prices.

Also, synthetic benchmarks and graphs geared towards gaming stats are not the end all/be all of the computing experience, therfore please remember that some folks use their computers for video production, photo editing/manipulation, database, scientific calcs etc., so some indications of suitability for those tasks would be helpful as well

 

[crazyeyesreaper]

In future reviews of systems I am running Specviewperf 13 which showcases performance for a multitude of various professional workloads. to go with the various current gaming and other tests. I can also test with different games easier so as new titles come out I can swap in and out as needed.

 

[crazyeyesreaper]

bump

 

[the54thvoid]

Many voices have spoken!

Yeah, surely, bench the system on its own merits. I don't think its like components where we look at reviews and shop around. A complete system is made from so many disparate components, it's too hard to compare against others. I think you ought to take the bull by the horns and do as you suggested yourself. Update the bench suite to suit current trends but only focus on the system at hand.

It's the logical option when you don't have access to a host of competing pre-builds.

 

[phanbuey]

I mean the whole point of benchmarking is to let people know how configurations stack up against one another. But especially when it comes to gaming there are really only 3 components that matter - CPU, Ram, and video card.

It's not like there are these infinite combinations. So it would be nice to have "THE BENCH" which is your standard mid to high end gaming rig vs whatever you're reviewing.

You can add noise, aesthetics, and build quality on as well - as those three are the primary things that set the pre-builds apart.

Ultimately people read benchmarks to see 1) if they should upgrade and 2) what they should upgrade to. It's very helpful to have either a comparison of more than one system across the benchmark suite, or to have benchmarks users can download and run (say timespy, cinebench, specific game benchmarks etc.) available to compare vs their home systems.

 

[rtwjunkie]

It's not like there are these infinite combinations.

Lol, actually, there practically are. Look at all the GPU’s W1z tests and people still complain about this card or that card not being included. Now make it combos of CPU, GPU and RAM and it’s a mess. I had to laugh because the first thing I thought of were the large number of combinations.

I maintain that when looking at a system, people want to see how the system seems to work to do what they need it for, and comparisons are not really important.

 

[phanbuey]

Lol, actually, there practically are. Look at all the GPU’s W1z tests and people still complain about this card or that card not being included. Now make it combos of CPU, GPU and RAM and it’s a mess.

I maintain that when looking at a system, people want to see how the system seems to work to do what they need it for, and comparisons are not really important.

W1zz's GPU graphs are some of the most informative out there because he includes so much. If he just reviewed each card and went - well this one games OK @ x resolution - no one would read those reviews.

You need at least a frame of reference for the review to be relevant. So either one reference point, or give out the benchmark suite to multiple TPUers and have them run it on their systems to provide additional data. Leverage the community. Sure it will throw some of the control out the window, but it's better IMO than having just a solo set of benchmarks out there and a conclusion of "this system can also game and content create" (which will be your conclusion 99.9% of the time with modern prebuilts).

Example with comparison : Walmart using cheapo cooling and single channel ram on their prebuilts, makes them perform X% worse than if you went with the same priced system from cyberpower... OR Walmart's system is X% slower than the average similar configured system of rtwJunkie at stock, and much slower when overclocked, so in this case it seems you're better off building one yourself.

Example without comparison: Walmart's system does take some shortcuts but it ran admirably through our test suite and can adequately stream and content create. Temps were fine and we didn't notice any throttling or major FPS issues in our test suite.

Which conclusion would you rather read after looking at pages of benchmarks?

 

[crazyeyesreaper]

W1zz's GPU graphs are some of the most informative out there because he includes so much. If he just reviewed each card and went - well this one games OK @ x resolution - no one would read those reviews.

You need at least a frame of reference for the review to be relevant. So either one reference point, or give out the benchmark suite to multiple TPUers and have them run it on their systems to provide additional data. Leverage the community. Sure it will throw some of the control out the window, but it's better IMO than having just a solo set of benchmarks out there and a conclusion of "this system can also game and content create" (which will be your conclusion 99.9% of the time with modern prebuilts).

Example with comparison : Walmart using cheapo cooling and single channel ram on their prebuilts, makes them perform X% worse than if you went with the same priced system from cyberpower... OR Walmart's system is X% slower than the average similar configured system of rtwJunkie at stock, and much slower when overclocked, so in this case it seems you're better off building one yourself.

Example without comparison: Walmart's system does take some shortcuts but it ran admirably through our test suite and can adequately stream and content create. Temps were fine and we didn't notice any throttling or major FPS issues in our test suite.

Which conclusion would you rather read after looking at pages of benchmarks?

What you fail to take into considering is whatever system I review, Likely has a GPU w1zzard already reviewed, a motherboard likely already reviewed, memory thats likely been reviewed, an SSD thats likely been reviewed. With each component reviewed in a more controlled environment that means systems need to be kept on hand. The other fact is where do I store all these systems even the ones I get to keep for extended periods have to be returned. That means keeping giant boxes and shipping crates as well.

The fact remains, I am not and will not constantly pay for parts to make people happy. Manufacturers don't want to send off $3500 systems for people to keep. Thats the reality. So either I buy this stuff outright to make people happy or comparisons remain stagnant because i can't effectively move away to newer benchmarks when warranted. While some companies are nice enough to let me keep systems for extended periods. That is not always the case. In fact its the exception to the rule. Suffice to say theres no really GOOD choice here to be honest.

Also who is to say other TPU employees have the same games? maybe they change there hardware frequently? Maybe they have their own reviews to handle, work regular jobs and don't want to run 20-40 benchmarks with what little free time they have. Its a shitty situation all around. The fact remains if a CPU underperforms in a reviewed system ill point that out, its not like we don't cover CPU reviews. same goes for GPUs or other oddities that come up. Leverage the community okay so a bunch of misc systems cluttering a chart. Previously I had mini-pc / laptops / desktops in the same charts. It was a mess. I can't see anyone on the forum running all those tests in a controlled manner.

You also act like me the reviewer isn't aware of said shortcomings such as single channel ram etc. I point it out religiously in laptop reviews because it does in fact hamper performance. Quality issues and design choices are talked about when the system is getting torn down. All of which remains in the reviews regardless of which path I choose for said reviews.

 

[EarthDog]

The individual system. As noted it's tough to compare as chances are you dont get like systems in the first place. The problem with that method is a lack of context though. It is a no win... but due to desktops having ungodly amounts of configs, readers should know there isnt going to be a direct comparison (like with GPUs, CPUs, etc).

Also, just because you test the individual system doesnt mean you need to bench more and have a comparison. The results are what they are. And if you change your test suite in year or so to keep up... it isnt a big deal. It's a whole system...its impossible to compare like to like unless you do a roundup asking for specific parts.

The key to systems is to share what sets it apart. Price, build quality/appearance, features, noise, etc since chances are people arent buying the same config.

 

[crazyeyesreaper]

The individual system. As noted it's tough to compare as chances are you dont get like systems in the first place. The problem with that method is a lack of context though. It is a no win... but due to desktops having ungodly amounts of configs, readers should know there isnt going to be a direct comparison (like with GPUs, CPUs, etc).

Also, just because you test the individual system doesnt mean you need to bench more and have a comparison. The results are what they are. And if you change your test suite in year or so to keep up... it isnt a big deal. It's a whole system...its impossible to compare like to like unless you do a roundup asking for specific parts.

The key to systems is to share what sets it apart. Price, build quality/appearance, features, noise, etc since chances are people arent buying the same config.

Thats basically my thought and thus review the system as it is. I can swap games and tests as necessary. In fact not having to worry about other systems means its easier to expand testing. Example the next system review that goes live will have Specviewperf 13 performance. I can focus on the extras (warranty, build quality, features, price to performance) its not like I don't spec out a similar system once its arrived to see where a custom built would be. I always verify the difference in price. So most bases are covered. and again everyone knows how a 9900K or a 2080 Ti performs. If those parts underperform in the review it would be noted and discussed. The doom and gloom is kinda funny.

 

[phanbuey]

What you fail to take into considering is whatever system I review, Like has a GPU w1zzard already reviews, a motherboard likely already reviewed, memory thats likely been reviewed, and SSD thats likely been reviewed with each component reviewed in a more controlled environment.

Great - include that data in the review.

The fact remains, I am not and will not constantly pay for parts to make people happy. Manufacturers don't want to send off $3500 systems for people to keep. Thats the reality. So either I buy this stuff outright to make people happy or comparisons remain stagnant because i can't effectively move away to newer benchmarks when warranted. While some companies are nice enough to let me keep systems for extended periods. That is not always the case. In fact its the exception to the rule.

I never suggested that.

Also whos to say other TPU employees have the same games? maybe they change there hardware frequently? Maybe they have their own reviews to handle work regular jobs and don't want to run 20-40 benchmarks with what little free time they have. Its a shitty situation all around. The fact remains if a CPU underperforms in a reviewed system ill point that out, its not like we don't cover CPU reviews. same goes for GPUs or other oddities that come up.

That's not what I am saying at all. Look at the sheer number of "Post your X benchmark scores" in the forums -- TPUers are a benching community -- we bench for fun. That data is already out there there are people maintaining those threads - W1zz's data is already out there. A good review will leverage that data and put the comparative performance current system that's being reviewed in context and present it (hopefully - with minimal work for the reviewer since the data is there already). I think presenting that will add quite a bit to any system review, and omitting it is a missed opportunity.

I'm not saying your reviews are bad or that you're lazy (at all). I sincerely hope you're not taking it that way. I'm just an avid review reader across a wide variety of sites, and to me, great system reviews are:

- Opinionated, as in the expert isn't afraid to point out things they loved/didn't like without fear offending X camp.

- Comparative - giving a good understanding of where the system fits in terms of the other options available. Analyzing how these parts work together vs say W1zz's 8700K review rig. Whether it's worth the money... Putting up some contextual performance numbers vs other systems.

- Detailed - going over sound levels, aesthetics, coolness factor etc. whatever metrics you're passionate about as a reviewer. I remember one guy reviewing based on subjective "smoothness" in-game experience, another one talking about living space and taking the rig to lan parties - those are memorable bits.

Bad reviews tend to be (again to me as a reader):
Uninformative - not many benchmarks, ones that are there are run at custom settings, by themselves, with nothing to compare to so the reader doesn't really know how the system performs. No discussion of quality of life elements.

Politically Correct - the review doesn't take a stand on whether this is a good machine or a bad machine; doesn't talk about the mix of hardware etc. i.e. pairing a 2080ti with a 2990wx for a gaming machine. Doesn't say whether you should or shouldn't buy this.

Generic - Long and at the end of the review you feel like you really only got the basics and are googling around for more information.

At the end of the day, running specviewperf 13 on a system by itself tells, me as a reader, very little. Plugging it into the "Post your cinebench score" ranking will tell me a WHOLE lot. Like hey - it's as fast as a highly overclocked 2700x... that's another option.

 

[EarthDog]

I don't spec out a similar system once its arrived to see where a custom built would be. I always verify the difference in price. So most bases are covered.

What does this mean? You don't spec it out, but verify the difference in price, somehow, without specing it out in other places?

As a random data nugget, I do believe that pricing it out by parts adds value to a review and takes mere minutes to figure out. It can show, at a high level, who is making more off the hardware (I mean granted there are a lot of other variables). What I think you will see is if you review, say 3 ibuypower boxes versus brand Y, perhaps IBP is closer to the DIY levels.

 

[crazyeyesreaper]

What does this mean? You don't spec it out, but verify the difference in price, somehow, without specing it out in other places?

As a random data nugget, I do believe that pricing it out by parts adds value to a review and takes mere minutes to figure out. It can show, at a high level, who is making more off the hardware (I mean granted there are a lot of other variables). What I think you will see is if you review, say 3 ibuypower boxes versus brand Y, perhaps IBP is closer to the DIY levels.

Voice to text fail. Suppose to read "You don't think i spec it out, verify the difference in price."

Straight from Corsair Vengeance review

That said, price to performance, while not terrible, is still not this system's strong suit. Enterprising DIY enthusiasts can easily build a system with similar specifications and the same keyboard and mouse for around $1900, which is nearly $500 less. However, having looked at what's available, it would mean a cheaper case, no RGB LEDs, etc. Building the same system with the same parts and a couple of RGB LED strips would run users around $2100. Keep in mind that these prices are without tax and shipping. Still, with the same parts, it's roughly a $300 upcharge for the Corsair system. However, considering how good it looks and how well it performs, that isn't too bad.

Literally i looked up pricing on every component in the system or I choose a similarly specced equipped replacement part to check DIY price etc for every desktop thats reviewed.

 

[EarthDog]

Cool. That is good!

I dont typically read system reviews as i am not a buyer or ever in that market... none here yet (sorry!). I reviewed one for anandtech in 2018...but it was more of a pilot for systems and didnt go past that one.