How to choose the correct GPU Card?

[danar jamal]

Hi All.

I need your help to choose the correct GPU and the correct Server for (Machine Learning, Visualization, and Data Streaming).

Thank you

 

[Athlonite]

I think you're going to be looking at Pro versions of GPU accelerators like one of the
AMD https://www.techpowerup.com/gpu-specs/radeon-instinct-mi100.c3496
or
nVidia https://www.techpowerup.com/gpu-specs/rtx-a5000.c3748

basically the MI series from AMD or the RTX A series are the sorts of GPU based accelerators that you're after for the typ of work you want to be doing

 

[danar jamal]

I think you're going to be looking at Pro versions of GPU accelerators like one of the
AMD https://www.techpowerup.com/gpu-specs/radeon-instinct-mi100.c3496
or
nVidia https://www.techpowerup.com/gpu-specs/rtx-a5000.c3748

basically the MI series from AMD or the RTX A series are the sorts of GPU based accelerators that you're after for the typ of work you want to be doing

Dear Athlonite,

Thank you for your reply.

One last question, we have been using A40 previously at other locations, and now we are looking to cut down the cost, do you think that A6000 can be a good fit for the use cases I have mentioned before? or we can go with T4, or RTX4090? and why ?

Thank you again.

 

[Calenhad]

A40 and A6000 are basically the same hardware. A40 is a Tesla card, a pure compute card with no display outputs. The A6000 is a Quadro so there are outputs. The GPU itself is the same model, so the performance is pretty much identical.

The RTX 4090 have higher raw compute power. But it has half the memory and no NVLink support. It is also a consumer card so it is not really designed to run at a max sustained load for weeks/months.

Your use case is rather vague though. A laptop with a GPU can run "Machine Learning, Visualization, and Data Streaming" tasks. It just depends on scale. But, since you mention using A40 at other locations, I understand that a laptop is not an option. I can't tell you if a 4090 will do though, since I have no idea of memory constraints for your planned tasks. Cooling situation can also be of importance

 

[Athlonite]

@Calenhad Pretty much took the words from my mouth but I'll also add that if you are using an accelerator for business purposes which you make money for then cost should not be a major function in your decision on which GPU based accelerator to buy as you should be able to write most of it off in tax rebates.

If you want to do the job properly then use the right tools for the job ie: don't be trying to mix concrete with a kids plastic sand bucket and spade (RTX4090) when you need a concrete mixer (A series or better GPU)

 

[dgianstefani]

A6000 Ada is more powerful for compute than 4090 as it has more cores.
A6000 Ampere is probably a lot cheaper, but may be slower/faster than RTX 4090, depending on the software you use. In most cases it will be slower, some specialist software it will be much faster.

 

[danar jamal]

A40 and A6000 are basically the same hardware. A40 is a Tesla card, a pure compute card with no display outputs. The A6000 is a Quadro so there are outputs. The GPU itself is the same model, so the performance is pretty much identical.

The RTX 4090 have higher raw compute power. But it has half the memory and no NVLink support. It is also a consumer card so it is not really designed to run at a max sustained load for weeks/months.

Your use case is rather vague though. A laptop with a GPU can run "Machine Learning, Visualization, and Data Streaming" tasks. It just depends on scale. But, since you mention using A40 at other locations, I understand that a laptop is not an option. I can't tell you if a 4090 will do though, since I have no idea of memory constraints for your planned tasks. Cooling situation can also be of importance

Thank you for your reply Calenhad,

In this case, I believe we either should go with 8X A6000 or 8XA40. is there any other option you suggest for this setup?

@Calenhad Pretty much took the words from my mouth but I'll also add that if you are using an accelerator for business purposes which you make money for then cost should not be a major function in your decision on which GPU based accelerator to buy as you should be able to write most of it off in tax rebates.

If you want to do the job properly then use the right tools for the job ie: don't be trying to mix concrete with a kids plastic sand bucket and spade (RTX4090) when you need a concrete mixer (A series or better GPU)

Dear Athlonite,
Thank you for your reply.

Actually, this will be a testbed to try out different scenarios. Is there any other option you may suggest? I know it is hard to choose the right GPU as we don't know the exact scenarios and applications.
do you think using (8x Tesla T4) can be an option?