Should we remove numeric scores for reviews?

[rtwjunkie]

They don't.

Only members of the forum are able to vote, which is a very specific group - more interested in gear and, most importantly, willing to spend more time learning about gear. It's plausible that a simple rating system is of less interest to them.
Moreover, people that voted are active members of the forum, so it's likely they don't even represent the community very well.

You guys have to decide if reviews are for the general public, which also includes people that aren't very interested in PC parts. They might even not understand a large portion of a review.
People want to buy a GPU. They open a site they trust, they read a few reviews and they compare.

I'm sure you're sometimes reading reviews (or user ratings at online stores) of stuff that you have to buy, but aren't that interested in. Cars, bikes, kitchen stoves, backpacks, women's perfume etc. Do you find ratings useful then?

You actually made a great point here I admit not Considering, in that the reviews are for the publisher c, many who know very little about the product.

 

[cadaveca]

I'm not against TPU grading products with an easy-to-use number, but what the fuck is the point of a site that literally seems unable to review a product badly, even when the conclusion of the same review clearly outs that product as being utter garbage?

So, when A product has issue, our normal response is to contact the manufacturer and ask why its broken. This may result in the issue being fixed, or the sample being recalled. I've had boards and memory not work, and they've been replaced with working ones.

We also do not test absolutely every product out there. That's why I can confidently say, that if it is not on these pages, don't buy it. What I am really saying is that if it doesn't make these pages, it must be garbage.

Ratings of washing machines are not useful for you? When you need a one, you're quickly studying the field and becoming a washing machine expert?

Well, you know, I have schooling for electrical engineering, HVAC, and psychology. I work as a technical writer. I'm not an expert on anything by any means, except perhaps specific PC parts after a decade of doing reviews (and I still wouldn't call myself an expert on any of that, either), but I do have a greater understanding on how mechanical and electrical things work than the average person. I don't pay tradespeople to fix broken stuff in my house; I do it myself, pay a journeyman when needed to sign off on it, and get things inspected as needed by insurance. I don't pay people to fix my cars, my PCs, my electronics... I do ALL of that myself, easily, without being an "expert". The only time I'm paying for anything of that sort of nature is when it's absolutely required. Look up what a technical writer is, and maybe you might understand why I do what I do... it's explaining how to solve problems.

The majority of other reviewers have zero education in the tech field. They are simply enthusiasts and gamers. My educational background is different than most, and that's why my opinion on some products can vastly differ from other reviewers... its coming from a completely different place and level of understanding.

You don't need to be an expert on anything to buy something. I mean, I've bought all sorts of junk through my life. Everything is built/made with planned obsolescence in mind, and it's no big deal to re-sell stuff you bought but were wrong about.

 

[notb]

Well, you know, I have schooling for electrical engineering, HVAC, and psychology. I work as a technical writer. I'm not an expert on anything by any means, except perhaps specific PC parts after a decade of doing reviews (and I still wouldn't call myself an expert on any of that, either), but I do have a greater understanding on how mechanical and electrical things work than the average person. I don't pay tradespeople to fix broken stuff in my house; I do it myself, pay a journeyman when needed to sign off on it, and get things inspected as needed by insurance. I don't pay people to fix my cars, my PCs, my electronics... I do ALL of that myself, easily, without being an "expert". The only time I'm paying for anything of that sort of nature is when it's absolutely required. Look up what a technical writer is, and maybe you might understand why I do what I do... it's explaining how to solve problems.

Yeah, you're just awesome. And humble, too. ;-)
The question stands. How do you buy women's perfume?

 

[VSG]

That said, what's the point of a reviewer "grading their experience" if they use a scale that either makes no sense or a scale that doesn't seem to have any real meaning?

There has *never* been a Motherboard review on TPU that received a numerical score of less than 6.5.
No graphics card has ever recieved below a 7.
No case has ever received below a 6.2.
No PSU has ever received below a 7.3
No headphones have received below a 7.9
No CPU cooler has ever received below a 6.5
No RAM has ever received below a 7
No keyboard has ever received below 6.8
No "Other" product has received below 7.2
No Mouse has received below a 6.
No SSD has received below a 7.5
No NAS has received below a 7.5
No external HDD has received below a 7.2
No CPU has received below a 7
No USB stick has received below 7.4
No mouse pad has received below a 7.3


I could keep going through all of the categories of review TPU has done, but I think you get the point by now.

In fact, the lowest score TPU has *ever* assigned to a product was a 6, an honour given to this waterblock in 2017 -
https://www.techpowerup.com/reviews/Bitspower/Summit_EF-X/
and to this mouse in 2007 -
https://www.techpowerup.com/reviews/OCZ/Equalizer/

The conclusion of that waterblock review reads "There are some silver linings in this dark cloud" and "Overall, even without the QC issues considered, there are simply too many options available that cost less and perform better. As such, I can not recommend the Bitspower Summit EF-X as it is now."

The first sentence of the mouse review conclusion is "The OCZ Equalizer disappoints in almost every possible way, right from the tracking performance to the driver setup. "

I'm not against TPU grading products with an easy-to-use number, but what the fuck is the point of a site that literally seems unable to review a product badly, even when the conclusion of the same review clearly outs that product as being utter garbage?

So I am the author of that waterblock review, and I can tell you that my basis for a 50% score is whether the product works and meets the minimum standards for the category. That block did so, and thus automatically got 5/10. The other point was for the customization options available, and then it got nothing for relative pricing, relative performance, and had nothing distinctly unique either. I feel that 6/10 was fair in that case.

You've based this opinion on an assumption that the rating scale has some possible values below 6.5 (like 0-10). Maybe it doesn't?

Or maybe it's just very unlikely for a product to get <5?
You know... if "5" means that something just works, but is unattractive and very mediocre, then scores below 5 would be for products that don't work.

Like, you know, Chinese knockoff keyboards that disintegrate after 2 months.
But TPU is not reviewing such products - they mostly review high-end, expensive (almost luxurious) gear.
Let's look at the keyboards, since I've already mentioned them. There is no review to date of something really cheap and basic - like A4Tech.
There's only a single Logitech keyboard - CRAFT (the most expensive one they have today) - despite Logitech being the most popular brand and having many gaming models in the lineup.

tl;dr
TPU doesn't really cover bad products, so why would they have to give bad marks?

The very first keyboard I reviewed was a Bloody keyboard which is an A4Tech brand, and I have since covered that brand again. I have also covered more obscure and also inexpensive keyboards and brands from Asia and elsewhere than anyone else I know, so please do look carefully before making such comments. As far as Logitech, I have approached them multiple times and never heard back from Logitech-G (the gaming business unit) but only heard back from Logitech (which is another unit) and hence the review of the CRAFT. That said, things are changing and I will have some Logitech-G keyboards in soon. No luck with Razer despite multiple attempts, so don't hold your breath on that.

 

[cadaveca]

Yeah, you're just awesome. And humble, too. ;-)
The question stands. How do you buy women's perfume?

I don't. My wife doesn't do fakeness like smells and face paint. We deal in being real and she smells just awesome without it. I mean, we have four kids...

If I wasn't a narcissist, I wouldn't think my opinion on PC parts was valid and be writing reviews that are contrary to what other people write. I wouldn't be voicing my opinion in a public forum thinking I knew better than everyone else. I was raised by a religious cult that thought I was the anti-Christ and dated my half-sister too, because no one was honest with me about why my parents split up when I was 2 and a 1/2 and I was drawn to this girl that everybody pretended didn't exist and ended up being my bloody sister. I am far from "normal". I have no problem owning who and what I am, so... how is any of that relevant and what's your point?








I just want people to love me...

 

[R-T-B]

I dont know about elsewhere in the world but in the States when in school 70% or 7/10 and below is a failing grade just FYI. 6.9 = failure in the States. Then again removing scores and doing as other sites do means 50 different awards that are just as worthless.

Not when I went to highschool, at least. Did things change? Graduated 2006.

 

[R0H1T]

Well this went Antarctic pretty quickly. I guess a lot of us need a vacation, especially from the cult of TPU & that includes me ~ before someone gets offended.

 

[notb]

The very first keyboard I reviewed was a Bloody keyboard which is an A4Tech brand, and I have since covered that brand again.

"Hey look, it's a true people's car from Volkswagen. It's called Veyron Super Sport"

Oh come on...!
Bloody is a very expensive brand. You've reviewed 3 Bloody keyboards on TPU - two with MSRP $150 and one with $130. Pfff.
The word "A4Tech" doesn't even appear on Bloody's website - apart from being mentioned in "Events" section (however, it's abundant in the source).

Bloody is not what people mean when talking about A4Tech.
They mean this:

I have also covered more obscure and also inexpensive keyboards and brands from Asia and elsewhere than anyone else I know, so please do look carefully before making such comments.

Almost everything is made in Asia, so I don't see the point of this remark.
Sadly, there is no way to sort products by MSRP in the review section. I believe the cheapest product there is this Genius:
https://www.techpowerup.com/reviews/Genius/SlimStar_8008/7.html
To be honest, I'm surprised you gave it 8.3, but whatever.

Anyway, many keyboards on the list are $100+ models and most are $50+. That's hardly cheap. ;-)

Based on how productivity-conscious reviews got lately and how suddenly everyone are after "best value" CPUs, I'm somehow surprised TPU isn't covering more mainstream peripherals.
It's not like everyone is using mechanical gaming keyboards. Just saying. ;-)

 

[VSG]

"Hey look, it's a true people's car from Volkswagen. It's called Veyron Super Sport"

Oh come on...!
Bloody is a very expensive brand. You've reviewed 3 Bloody keyboards on TPU - two with MSRP $150 and one with $130. Pfff.
The word "A4Tech" doesn't even appear on Bloody's website - apart from being mentioned in "Events" section (however, it's abundant in the source).

Bloody is not what people mean when talking about A4Tech.
They mean this:

Almost everything is made in Asia, so I don't see the point of this remark.
Sadly, there is no way to sort products by MSRP in the review section. I believe the cheapest product there is this Genius:
https://www.techpowerup.com/reviews/Genius/SlimStar_8008/7.html
To be honest, I'm surprised you gave it 8.3, but whatever.

Anyway, many keyboards on the list are $100+ models and most are $50+. That's hardly cheap. ;-)

Based on how productivity-conscious reviews got lately and how suddenly everyone are after "best value" CPUs, I'm somehow surprised TPU isn't covering more mainstream peripherals.
It's not like everyone is using mechanical gaming keyboards. Just saying. ;-)

I can only cover brands willing to send products in for review since TPU does not buy products, and A4Tech sent their Bloody brand keyboards. I have plenty of other mechanical and non-mechanical keyboards covered, or soon to be covered, which fall in the sub $50 range (Aukey, 1stPlayer, Genius and more). Again, it comes down to the marketing budget of companies and usually the less expensive products don't have a budget for sampling

 

[sneekypeet]

usually the less expensive products don't have a budget for sampling

Or they use the buy my product off Amazon, do a "review" on Amazon, and we will refund the cost, marketing strategy.

 

[R-T-B]

But you aren't the majority. The forum users are far from it. You're the vocal minority, by definition.

People forget this.

I for example, am just one little frog like man (or man like frog, same thing really, the point is one), but I make a very disproportiante amount of noise to my size.

Yes, I might even be proud of that fact. But it does not make me or even this forum the "majority." The majority likely doesn't even DO the polls at all

I just want people to love me...

It's too late. You had that goat thing in your avatar for far too long, and no one likes having a reputation of being in a relationship with a goat.

Then again, I'm also screwed, based on this priniciple.

Hold me Dave...

 

[crazyeyesreaper]

Not when I went to highschool, at least. Did things change? Graduated 2006.

In Maine schooling from Elementary through high school followed the college grading method. So anything sub 70 is failure pure and simple.

For instance the Silentium PC review where it scored sub 7.0. The cooler worked fine for stock settings but the base design and likely pressure isn't high enough from what I saw. Resulting in sub par performance on the newer 8000 series. Especially considering its massive size where single tower designs half the weight are on par or beat it. While the price is okay for the performance. Its a very sub optimal cooler. How do you give something a 4/10 or 5/10 if it cools just fine by default. It mounts theres no damage or problems? If a product works it basically passes in my opinion. However this one sample was an outlier. While not all coolers are amazing. The majority of work just fine if you bought them off a shelf they would do the job. Obviously others work better as well. But it also comes down to availability.

Another example is the 360 mm AIOs as of late. with 7.X scores. They work they pass all the tests but a 240 does a better job. As we move towards RGB fans airflow has been reduced performance falls so you lose performance in favor of pretty lights. But since the market is buying this shit in droves its working. People care less about performance these days than they do how good something looks. This forum and its regular posters are the exception.

A score is helpful but then again no one typically reads the reviews anyway. Most just check the graphs see where something winds up checks for the price and thats it. Sad fact of life. If you rate a product based on quality / performance etc and give it a good score people get pissy cause its not as cost effective as a custom built system etc. Can't make everyone happy. At this point I don't care if the scoring system stays or goes.

However without a quick look score I can bet traffic for the site would likely drop.

 

[cadaveca]

However without a quick look score I can bet traffic for the site would likely drop.

I like THAT bet.

 

[R-T-B]

In Maine schooling from Elementary through high school followed the college grading method. So anything sub 70 is failure pure and simple.

Hmmm... that seems unusual to me and I was schooled in both California and Washington, but irrelevant to the point you were trying to make (there, 60% and above is passing, 70% is "average", etc)

Don't get me started on my college, evergreen. They basically didn't grade at all. But then, they are different, to say the least.

 

[FreedomEclipse]

I feel the review score is inflated. Even lackluster GPU’s for instance rarely go below 8 -something. The vast majority are over 9. I actually ignore the score now, because it is meaningless.

Why not a “Not recommended”, “Approved” or “Good” seal at the end? Save something like “W1zzard’s Seal of Approval” for exemplary products? It would necessarily have to be rarely given to mean something.

Just my 2 cents.

agree with you i do. Personally for me, the scores became meaningless years ago but the basis of the reviews were still great so i didnt really feel the need to open my mouth. I cant remember when exactly but Guru3D did away with the scoring years ago.. Infact i cant remember the last time they had actual scores on their reviews.

i voted NO but i think things need . . changed.

personally i like review scoring broke down into categories such as, value, performance, quality of components/build and maybe anesthetics (though subjective it will show people how the reviewer scores for RGB) then scores avg.

I think TweakTown does their breakdowns like this too. It can be quite informative like that

::EDIT::

Basically, what im trying to say is that the scoring/rating doesn't seem that relevant anymore in 2018. I strongly believe that just having 'award stamps' will open up peoples minds a little more to whats good and to make their own decisions and opinions based on the review rather than search for the 'best' which pretty much means every single GPU review in the database thats 9.0 and up.

Im not quite sure how to explain it further.

 

[Apocalypsee]

Truth to be told, I never bother about the scores because there is pros and cons for that. I look at the pros and cons and made my own judgement from that (e.g RGB dont matter to me so the cons that say no RGB not applicable to me)

The only thing that made me read the score when people talk about it in the forum (e.g scores too high but have big cons etc.)

 

[apertotes]

I know it's late, but I wanted to say that I indeed started to spend more time on the reviews since the change. In fact, I spent a lot of time looking for the numeric score. I tried with different browsers, incognito mode, I tried on my phone, and at the office. I alone may be responsible for all the spike in traffic you may have noticed on conclusion pages.

Finally, I made a Google search and I arrived here.

So, now that I know what has happened I can say for sure that I won't spend that much time looking for the score anymore and that the usefulness of reviews has decreased for me. I am somewhat savvy and I know the difference between a ROM and a DAC, but don't push me much more. I often get lost while reading the pros and cons and having a single numeric summary from an expert made it much easier to compare potential purchases.

However, I agree that using a 10 point scale when nothing goes below 7 is useless. It may be heartbreaking, but I believe that you should use low numbers for the worse products you review, even if they are not bad products. At least that way we would compare 3s with 8s and not 9.7s with 9.5s. I believe that one of the responsibilities of a reviewer is to establish clear hierarchy of competing products. Using such a narrow scale does not help, and removing the score completely even less.

I am not rage quitting at all. I am still going to read most of your reviews. But I think you should know that I will miss the score and won't get used to not having it.

 

[FordGT90Concept]

However, I agree that using a 10 point scale when nothing goes below 7 is useless. It may be heartbreaking, but I believe that you should use low numbers for the worse products you review, even if they are not bad products. At least that way we would compare 3s with 8s and not 9.7s with 9.5s. I believe that one of the responsibilities of a reviewer is to establish clear hierarchy of competing products. Using such a narrow scale does not help, and removing the score completely even less.

That's precisely why it was removed. The pros and cons as well as the conclusion text at large are a better gauge of comparting products than the score. For example, if power consumption is important to you and it lists high power consumption as a con, you should probably keep looking for a different product. A simple numeric score doesn't provide that kind of context sensitive scoring.

What you were suggesting lines up with what crazyeyesreaper said above with applying a school grade scale where anything below 70% is failure--letter grades assigned above that. I think that's better than the 10.0 scale used before but I'm not sure I wouldn't just disregard it too.

 

[Vayra86]

But I think you should know that I will miss the score and won't get used to not having it.

And you should know that the metric you have always used (score) to determine how good something is, was the wrong one and it always will be. Even on websites that do use the entire 1-10 scale.

That isn't exclusive to TPU. Literally every reviewer that uses a scale like this has the same issues with it, for all the obvious reasons. Nobody is completely without bias. A pro-con list is the best most people can achieve to stay away from it.

Scoring systems can work, but they require big data. Here's an example of how they DO work (and then only when a title is fed with a high number of scores):

https://www.metacritic.com/

 

[Aquinus]

In Maine schooling from Elementary through high school followed the college grading method. So anything sub 70 is failure pure and simple.

Just jump the border here to NH and it turns back into "F" means "Failure" and >= 60% is technically passing.

Either way, I think the point is if bad hardware ends up getting sent back or recalled, most reviews are going to end up "passing," so what is the point of having 0-6 when most reviews get 7-10? The problem is that the number is incredibly subjective. It's not like there are criteria (at least exposed to us,) that say what factors contributed to a product getting an 8.5 or something like that. I can't do some math and reproduce it and to me, that's a problem.

If we're going to include numbers, I want to know how they were derived. If it's a person just making up whatever feels right, then we need to get rid of it because it literally means nothing for just about anything.

So, maybe if you guys created a formula for how products get their score generated that's consistent and based off gathered data, then I'm all for it. Otherwise, it means nothing to me.

 

[EarthDog]

It's gone already boys...https://www.techpowerup.com/forums/threads/where-are-the-numeric-scores.248663/


....another reason why it would have been "beneficial" to share the decision with the user base instead of just implementing it.

 

[Aquinus]

It's gone already boys...https://www.techpowerup.com/forums/threads/where-are-the-numeric-scores.248663/


....another reason why it would have been "beneficial" to share the decision with the user base instead of just implementing it.

A great suggestion might be to lock this thread and leave a final post and update the first post explaining where this went because if I hadn't read that link just now, I wouldn't have known.

 

[Flyordie]

I'm just gonna say this- The numeric score system only would really work if it was done in comparison to a like-minded product. Like how different engines have different HP/Torque ratings.

Its gotta have a solid basis in repeatable statistics or else over time the water just gets muddy.

 

[apertotes]

So, all those against numeric scores, you do not rate movies or shows on IMDB? You did not use stars on Netflix (may whoever got rid of them never find solace)? Don't rate apps on the Play Store or hotels on Booking.com?

As I said, the list of pros and cons only works for whose who can grasp all the technicalities. Whether that is the whole reader base of the site or not is something the admins have to ponder.

 

[Vayra86]

So, all those against numeric scores, you do not rate movies or shows on IMDB? You did not use stars on Netflix (may whoever got rid of them never find solace)? Don't rate apps on the Play Store or hotels on Booking.com?

As I said, the list of pros and cons only works for whose who can grasp all the technicalities. Whether that is the whole reader base of the site or not is something the admins have to ponder.

We do. Did you read my post? I said specifically: big data + scoring works well. That is also how the reviewing and rating systems work that you mention here.

The 1-5 star ratings from a userbase: if you see only one review or rating, you know its unreliable. If there are a few hundred all high rates, you know its a more reliable score. A TPU review score would only be N=1. Thus not reliable.