1. Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

What would you like to see in CPU reviews?

Discussion in 'Comments & Feedback' started by W1zzard, Mar 19, 2017.

  1. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    16,829 (3.45/day)
    Thanks Received:
    17,661
    Now that my first CPU review is posted, I'd like to ask the community for thoughts on methodology, benchmark selection etc. (This not to discuss anything Ryzen-specific, posts will be deleted).

    What applications do you use in daily life that make you wait? Would you like to see more/fewer games? Other resolutions?

    - Databases?
    - Rendering?
    - Simulation?

    I can automate nearly every application to perform benchmark testing, so don't worry if it can be done :)
     
    T-Bob, phanbuey, springs113 and 11 others say thanks.
    10 Year Member at TPU
  2. VSG

    VSG Super Moderator Staff Member

    Joined:
    Jul 1, 2014
    Messages:
    575 (0.49/day)
    Thanks Received:
    451
    Let's see if anyone has ideas for a reproducible test suite in Matlab, cEmu and something database/dataset related like R.

    What about a game benchmark combined with streaming it live?
     
    Last edited: Mar 19, 2017
    T-Bob, phanbuey, xkm1948 and 2 others say thanks.
  3. uuuaaaaaa

    Joined:
    Sep 2, 2011
    Messages:
    928 (0.42/day)
    Thanks Received:
    527
    Location:
    Porto
    As an heavy Matlab user I would love to have an idea of a cpu performance on such application. Since Matlab is paid and quite expensive, maybe Octave could be an alternative!
     
    Last edited: Mar 19, 2017
    nemesis.ie says thanks.
  4. Smanci

    Joined:
    Jul 7, 2016
    Messages:
    72 (0.16/day)
    Thanks Received:
    38
    Frame times.
     
    n-ster, EntropyZ, IceScreamer and 2 others say thanks.
  5. Outback Bronze

    Outback Bronze

    Joined:
    Aug 3, 2011
    Messages:
    718 (0.32/day)
    Thanks Received:
    320
    Location:
    At the Pub
    Well I'm after 4K reviews which is pretty much non existent. Since this is the way going forward.

    Apart from that it would be nice to see how crossfire and SLI fairs also. Perfect example is Ryzen.

    Obviously this is from a gaming point of view.

    BTW your list of games for the Ryzen review was first class!

    Thanks.
     
    Last edited: Mar 19, 2017
  6. nemesis.ie

    nemesis.ie

    Joined:
    Jan 17, 2006
    Messages:
    427 (0.10/day)
    Thanks Received:
    112
    Multiple broswer tabs with loading/switching etc. and between that and office-type apps.

    Maybe a "simulation" of a non-gaming scenario in an office where someone is moving back and forth between database screens, email etc. and maybe compiling some code or running tests in the background.

    Running tasks in the background while gaming, e.g. Skype on a second screen while you give someone on the video link a tutorial.

    Web pages on a second screen while gaming (e.g. monitoring social media, stocks etc.).

    Time to switch out of/close a game to the desktop ("boss mode"). ;)
     
    Last edited: Mar 19, 2017
    T-Bob, Sasqui, rtwjunkie and 2 others say thanks.
    10 Year Member at TPU
  7. BarbaricSoul

    BarbaricSoul

    Joined:
    Feb 21, 2008
    Messages:
    6,223 (1.78/day)
    Thanks Received:
    3,312
    Location:
    S.E. Virginia
    Personally, crunching is currently what's important to me when picking a CPU.
     
    T-Bob, stinger608, revin and 2 others say thanks.
    Crunching for Team TPU
  8. r9

    r9

    Joined:
    Jul 28, 2008
    Messages:
    2,515 (0.75/day)
    Thanks Received:
    512
    I was thinking about this since fx series came out.
    During benching your disable everything to make the test as valid and comparable as possible.
    But as nemesis.ie said that is not how we use our computers.
    Downloading stuff in background, maybe youtube on the second screen or stocks (whatever rocks your world), maybe some encoding.
    One test Gaming while running something else in the background would give great insight.

    In the Ryzen review you can see that the power draw on i7 7700k is the same for Prime95 and in gaming which means that in both scenarios is being utilized maximum.
    For comparison Ryzen had 154W Prime95 vs 121W Gaming which means its not utilizing the CPU fully in gaming.
    And I'm not trying to start a debate here for Ryzen, but benching should represent real world scenarios and for most people is running something else in the background while gaming.
    For people that gonna say that they are exclusively gaming only, unless you are disabling updates, Antivirus, Firewall etc you are running something else.
     
    Last edited by a moderator: Mar 19, 2017
    nemesis.ie says thanks.
  9. Norton

    Norton Moderator & WCG-TPU Captain Staff Member

    Joined:
    Dec 21, 2011
    Messages:
    12,279 (5.84/day)
    Thanks Received:
    35,009
    Location:
    USA
    ^^ This!

    However, since crunching relies on many factors it would be difficult to nail down actual crunching capability unless you were able to test it for at least 2-3 weeks.

    There is a CPU benchmark built into the BOINC Manager software that may allow some comparison. It's a quick test that measures the Floating Point/Whetstone and Integer/Dhrystone performance of the CPU on a per core/thread basis.

    These results are available online through various stats sites such as this one for an 1800X in use by one of our Team members:
    http://stats.free-dc.org/stats.php?page=host&proj=bwcg&hostid=3834496
    *see top half of the page- note that this website has been a little wonky lately :rolleyes:

    Example- This is an 1800X vs a 3930k
    * Note- not a completely accurate comparison since the benchmark varies between operating systems...

    [​IMG]
     
    Last edited: Mar 19, 2017
    T-Bob, night.fox, stinger608 and 5 others say thanks.
    Crunching for Team TPU
  10. notb

    Joined:
    Jun 28, 2016
    Messages:
    861 (1.90/day)
    Thanks Received:
    216
    Location:
    Warsaw, Poland
    Well.. we've already exchanged a few posts, but I'm glad this thread was created. :)

    Whatever benchmarks you'll use, I do have 2 general suggestions.
    I. Let's keep stuff fairly mainstream, accessible and - ideally - make at least some of the benchmarks downloadable. This would make it possible for us to check how our machines compare with the newest stuff.
    As such, I think you should consider free or paid mainstream software only, so - for example - not things like Matlab, AutoCAD, Photoshop etc.
    Moreover, you could at least think about starting a "3rd party" community results' database. I'm not saying it's essential, but could work as an extra resource next to results in controlled environment.
    II. I'd love to see some more statistics in the results, but keeping it as simple as possible: FPS quantiles for gaming, number of samples and dispersion in time-measuring benchmarks etc.


    As for the benchmarks, here are some possibilities:
    1) IMO a serious Excel test is a must. Access could be incorporated (as today it's more an Excel back-end for data storage than a sensible database).
    Germany's population is 80 mio, so it's very likely that just in your country there are more people looking at Excel for >4h a day than those doing serious 3D rendering all over the planet. :)
    2) Database (ideally both SQL and NoSQL). This should include some advanced analytic coding and scripting (as basic data manipulation tests the storage speed more than anything else). You'll need a DB with decent scripting for that (PostgreSQL?). You could consider SQLite as well, because it's hugely popular as embedded db in a lot of software.
    3) Numerical computation / data analysis - huge amount of software to choose from: R, Octave, Python etc. This could include generating 2D images as a result (a complicated plot/graph can take a few seconds, so it shouldn't be underestimated :)).
    4) Virtualization - ability to work with multiple VMs (each with some CPU-heavy tasks).
    5) Everyday tasks / productivity - running a lot of things at the same time rather than separate. Such benchmark tests a lot more than just multi-thread performance. I'd go for a movie + compression + firefox + file copying etc. I'd exclude gaming, because it doesn't offer stable CPU load and would ruin the results.
    6) Antivirus scan - totally ignored in many reviews for a reason I don't understand. A weekly full scan is one of the most irritating things I do (even worse than off-site backup).

    I'm not suggesting video encoding and rendering for a simple reason - some very good benchmarks are already available.
     
    Last edited: Mar 19, 2017
    springs113 and VSG say thanks.
  11. LFaWolf

    LFaWolf

    Joined:
    Jul 9, 2016
    Messages:
    91 (0.21/day)
    Thanks Received:
    49
  12. Ralfies

    Ralfies

    Joined:
    Dec 6, 2012
    Messages:
    145 (0.08/day)
    Thanks Received:
    74
    Location:
    Oklahoma
    Although multi-gpu seems to be becoming less relevant, I too would love to see some benches in CPU reviews. It doesn't need to be a ton, just a couple of games that have good support (I don't own any 2016/2017 games so I can't recommend any).
     
  13. Sasqui

    Sasqui

    Joined:
    Dec 6, 2005
    Messages:
    9,498 (2.20/day)
    Thanks Received:
    3,549
    Location:
    Manchester, NH
    I think you're referring to reality? ;)
     
    10 Year Member at TPU
  14. LightningJR

    LightningJR

    Joined:
    Jun 25, 2010
    Messages:
    809 (0.31/day)
    Thanks Received:
    569
    Yeah some kind of crunching benchmark would be nice. I don't know the legality of this but it would be interesting to see some emulator results too since they are so dependent on CPU performance.
     
    revin says thanks.
    Crunching for Team TPU
  15. xkm1948

    xkm1948

    Joined:
    Mar 18, 2008
    Messages:
    2,334 (0.67/day)
    Thanks Received:
    1,393
    I will write up a small package for you to test CPU performance on genome assembly. Bioinformatics is a huge rising field where most computational resources go in most Universities. Not gonna be done fast though. My hope if to get a paper published on this along with starting something similar to Stanford Folding@home. Basically recruiting all these awesome idling HEDT processors to help assemble genome of various species.
     
    T-Bob, revin and nemesis.ie say thanks.
  16. cadaveca

    cadaveca My name is Dave

    Joined:
    Apr 10, 2006
    Messages:
    16,384 (3.92/day)
    Thanks Received:
    10,678
    Location:
    Parkland County, Alberta
    If CPU has IGP, test its performance. Without that, kind of half the chip isn't tested.
     
    T-Bob, Aquinus, nemesis.ie and 3 others say thanks.
    10 Year Member at TPU
  17. VSG

    VSG Super Moderator Staff Member

    Joined:
    Jul 1, 2014
    Messages:
    575 (0.49/day)
    Thanks Received:
    451
    Seconded.
     
  18. buzzi

    buzzi

    Joined:
    Jun 16, 2016
    Messages:
    67 (0.14/day)
    Thanks Received:
    75
    I'd like to see if gains from previous generation are worth buying
     
  19. jboydgolfer

    jboydgolfer

    Joined:
    Oct 17, 2012
    Messages:
    6,167 (3.42/day)
    Thanks Received:
    8,437
    Location:
    Massachusetts
    just Two words:

    The Division


    :cool:+:kookoo:=jboydgolfer:laugh:
    @Kursah
     
  20. HD64G

    HD64G

    Joined:
    Apr 30, 2011
    Messages:
    745 (0.32/day)
    Thanks Received:
    672
    Location:
    Greece
    Multitasking is a good thing to be tested, especially for CPUs having 8 or more threads. Gaming, streaming, virus scanning, compessing, video-encoding could be some proposed tasks to be included.
     
  21. natr0n

    natr0n

    Joined:
    Jan 29, 2012
    Messages:
    4,134 (2.00/day)
    Thanks Received:
    3,300
    Location:
    Boca Raton, Florida
    Handbrake
     
    xkm1948 and buzzi say thanks.
  22. OneCool

    OneCool

    Joined:
    Nov 27, 2005
    Messages:
    1,065 (0.25/day)
    Thanks Received:
    194
    Location:
    Look behind you!!
    Be nice to see a direct comparison core/core and thread/thread no matter the price difference.

    Also some dancing clowns would be nice :toast::nutkick:
     
    10 Year Member at TPU
  23. Grings

    Grings

    Joined:
    Nov 16, 2006
    Messages:
    2,717 (0.69/day)
    Thanks Received:
    446
    Location:
    Blighty
    I would like to see the gaming benches run on ALL configurations in the test

    The X99 setups in the Ryzen review are also not as fast as the 7700k in gaming, and the review should have reflected that as a first look at where the ryzen sits against intels 2 platforms
     
    10 Year Member at TPU
  24. Vayra86

    Vayra86

    Joined:
    Sep 17, 2014
    Messages:
    2,916 (2.65/day)
    Thanks Received:
    2,434
    Location:
    Duiven, Netherlands
    I liked the Ryzen review and its bench suite, good mix of titles in gaming and beyond.

    What I would like to see in a CPU test is some side-by-side work, being browser with youtube playing + CPU limited game. Or game recording + game.
     
  25. jormungand

    jormungand

    Joined:
    Dec 29, 2012
    Messages:
    327 (0.19/day)
    Thanks Received:
    207
    :roll::roll::roll::twitch::twitch::twitch:
     

Currently Active Users Viewing This Thread: 1 (0 members and 1 guest)