• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

How to choose the correct GPU Card?

danar jamal

New Member
Joined
Nov 29, 2022
Messages
3 (0.05/day)
Hi All.

I need your help to choose the correct GPU and the correct Server for (Machine Learning, Visualization, and Data Streaming).

Thank you
 
Joined
Jan 31, 2010
Messages
5,064 (1.07/day)
Location
Gougeland (NZ)
System Name Cumquat 2021
Processor AMD RyZen R7 5800X @4750MHz 1.298V
Motherboard Asus Strix x570-F Gaming
Cooling Cryorig R1 Universal + CM MasterGel Pro TP
Memory 16GB GSkill Ripjaws V Black 4000 MTs (2x8GB) @3866MT's
Video Card(s) Sapphire Nitro+ OC RX6800 16GB DDR6 2270Cclk / 2010Mclk
Storage 1x Adata SX8200PRO NVMe gen3 x4 1X Samsung 860 EVO 500GB, 12TB of HDD Storage
Display(s) AOC 24G2 IPS 144Hz FreeSync Premium 1920x1080p
Case SilverStone Raven RV02B-W + 3x SS AP181 + USB3.0 front panel upgrade
Audio Device(s) RX6800 via HDMI + Pioneer VSX-531 amp Technics 100W 5.1 Speaker set
Power Supply EVGA 1000W G5 Gold
Mouse Logitech G502 Proteus Core
Keyboard Logitech G815
Software Windows 11 X64 PRO (build 22H2)
Benchmark Scores it sucks even more less now ;)

danar jamal

New Member
Joined
Nov 29, 2022
Messages
3 (0.05/day)
I think you're going to be looking at Pro versions of GPU accelerators like one of the
AMD https://www.techpowerup.com/gpu-specs/radeon-instinct-mi100.c3496
or
nVidia https://www.techpowerup.com/gpu-specs/rtx-a5000.c3748

basically the MI series from AMD or the RTX A series are the sorts of GPU based accelerators that you're after for the typ of work you want to be doing
Dear Athlonite,

Thank you for your reply.

One last question, we have been using A40 previously at other locations, and now we are looking to cut down the cost, do you think that A6000 can be a good fit for the use cases I have mentioned before? or we can go with T4, or RTX4090? and why ?

Thank you again.
 
Joined
Feb 22, 2022
Messages
162 (0.47/day)
Processor AMD Ryzen 9 5950X
Motherboard Asus Crosshair VIII Dark Hero
Cooling Custom Watercooling
Memory G.Skill Trident Z Royal 2x16GB
Video Card(s) MSi RTX 3080ti Suprim X
Storage 2TB Corsair MP600 PRO Hydro X
Display(s) Samsung G7 27" x2
Audio Device(s) Sound Blaster ZxR
Power Supply Be Quiet! Dark Power Pro 12 1500W
Mouse Logitech G903
Keyboard Steelseries Apex Pro
A40 and A6000 are basically the same hardware. A40 is a Tesla card, a pure compute card with no display outputs. The A6000 is a Quadro so there are outputs. The GPU itself is the same model, so the performance is pretty much identical.

The RTX 4090 have higher raw compute power. But it has half the memory and no NVLink support. It is also a consumer card so it is not really designed to run at a max sustained load for weeks/months.

Your use case is rather vague though. A laptop with a GPU can run "Machine Learning, Visualization, and Data Streaming" tasks. It just depends on scale. But, since you mention using A40 at other locations, I understand that a laptop is not an option. I can't tell you if a 4090 will do though, since I have no idea of memory constraints for your planned tasks. Cooling situation can also be of importance
 
Joined
Jan 31, 2010
Messages
5,064 (1.07/day)
Location
Gougeland (NZ)
System Name Cumquat 2021
Processor AMD RyZen R7 5800X @4750MHz 1.298V
Motherboard Asus Strix x570-F Gaming
Cooling Cryorig R1 Universal + CM MasterGel Pro TP
Memory 16GB GSkill Ripjaws V Black 4000 MTs (2x8GB) @3866MT's
Video Card(s) Sapphire Nitro+ OC RX6800 16GB DDR6 2270Cclk / 2010Mclk
Storage 1x Adata SX8200PRO NVMe gen3 x4 1X Samsung 860 EVO 500GB, 12TB of HDD Storage
Display(s) AOC 24G2 IPS 144Hz FreeSync Premium 1920x1080p
Case SilverStone Raven RV02B-W + 3x SS AP181 + USB3.0 front panel upgrade
Audio Device(s) RX6800 via HDMI + Pioneer VSX-531 amp Technics 100W 5.1 Speaker set
Power Supply EVGA 1000W G5 Gold
Mouse Logitech G502 Proteus Core
Keyboard Logitech G815
Software Windows 11 X64 PRO (build 22H2)
Benchmark Scores it sucks even more less now ;)
@Calenhad Pretty much took the words from my mouth but I'll also add that if you are using an accelerator for business purposes which you make money for then cost should not be a major function in your decision on which GPU based accelerator to buy as you should be able to write most of it off in tax rebates.

If you want to do the job properly then use the right tools for the job ie: don't be trying to mix concrete with a kids plastic sand bucket and spade (RTX4090) when you need a concrete mixer (A series or better GPU)
 

dgianstefani

TPU Proofreader
Staff member
Joined
Dec 29, 2017
Messages
2,017 (1.09/day)
Location
Swansea, Wales
System Name Silent
Processor Ryzen 5800X3D @ 4.55ghz
Motherboard Asus Strix X570I
Cooling Optimus AMD Raw Copper/Plexi, EK Copper 240/40mm, Utilitube D5, 4x NF-A12x25, Mayhems XTR Nano White
Memory 32 GB Dominator Platinum RGB 3933mhz 14-16-16-30 w/2x NF-A4x10 fan
Video Card(s) Rtx 3080ti Founders (Kryonaut Extreme, 15W/mK pads, Corsair XG7 Waterblock)
Storage Optane DC P1600X 118 GB, Samsung 980 Pro 1 TB
Display(s) 32" 240 Hz 1440p Samsung G7 , 32" 1440p 60 Hz
Case Sliger SM570 Custom 13-Litre
Audio Device(s) Schiit Hel, Vmoda Crossfade M100 Master, Razer Nommo Pro
Power Supply Corsair SF750 Platinum
Mouse Razer Viper Pro V2 Mercury White
Keyboard Razer Huntsman Elite w/Mercury PBT keycaps
A6000 Ada is more powerful for compute than 4090 as it has more cores.
A6000 Ampere is probably a lot cheaper, but may be slower/faster than RTX 4090, depending on the software you use. In most cases it will be slower, some specialist software it will be much faster.
 

danar jamal

New Member
Joined
Nov 29, 2022
Messages
3 (0.05/day)
A40 and A6000 are basically the same hardware. A40 is a Tesla card, a pure compute card with no display outputs. The A6000 is a Quadro so there are outputs. The GPU itself is the same model, so the performance is pretty much identical.

The RTX 4090 have higher raw compute power. But it has half the memory and no NVLink support. It is also a consumer card so it is not really designed to run at a max sustained load for weeks/months.

Your use case is rather vague though. A laptop with a GPU can run "Machine Learning, Visualization, and Data Streaming" tasks. It just depends on scale. But, since you mention using A40 at other locations, I understand that a laptop is not an option. I can't tell you if a 4090 will do though, since I have no idea of memory constraints for your planned tasks. Cooling situation can also be of importance
Thank you for your reply Calenhad,

In this case, I believe we either should go with 8X A6000 or 8XA40. is there any other option you suggest for this setup?

@Calenhad Pretty much took the words from my mouth but I'll also add that if you are using an accelerator for business purposes which you make money for then cost should not be a major function in your decision on which GPU based accelerator to buy as you should be able to write most of it off in tax rebates.

If you want to do the job properly then use the right tools for the job ie: don't be trying to mix concrete with a kids plastic sand bucket and spade (RTX4090) when you need a concrete mixer (A series or better GPU)
Dear Athlonite,
Thank you for your reply.

Actually, this will be a testbed to try out different scenarios. Is there any other option you may suggest? I know it is hard to choose the right GPU as we don't know the exact scenarios and applications.
do you think using (8x Tesla T4) can be an option?
 
Top