Nvidia Tesla T4 cards incorrectly report 100% memory utilization

[bsee-ino]

GPU-Z reports 100% memory utilization for Tesla T4 cards. Monitoring the same card with Nvidia SMI reports the correct usage. Confirmed in GPU-Z v2.36.0 (latest). This is not a new issue.

 

[W1zzard]

Interesting, any chance I could use Remote Desktop or Teamviewer to check out the problem and try a few debug builds?

 

[StefanM]

Same issue as mentioned here:

Tesla T4 Problems

Hello, We have a Threadripper TRX40 workstation using Tesla T4 overheating. I had some troubles during the installation. I thought it was gone! But the problem seems like still remain unsolved. GPU temperature is at minimum 57, on simple benchmarks ( not eve stress test ) over heating up to 91...

www.techpowerup.com

 

[bsee-ino]

I can't give an remote session, sorry. This is being used for a business. However, the thread StefanM linked is exactly the same. This server is a Gigabyte, just like the motherboard in that thread. I'm not sure how GPUz queries the memory usage, so not sure if the motherboard is relevant at all. The issue occurred in multiple servers using both AMD & Intel cpus.

 

[W1zzard]

Oh, this is VRAM usage, sorry I assumed you meant "Memory Controller Load".

Looks like an overflow indeed, let me test on other cards with around 16 GB memory or more

Edit: tested on RTX 3090 (24 GB) and RX 6800 XT (16 GB) and works for me.

Which Windows version do you use? If Windows 10, which build?

 

[StefanM]

You can also double-check with task manager->performance->GPU

GPUs in the task manager | DirectX Developer Blog

The below posting is from Steve Pronovost, our lead engineer responsible for the GPU scheduler and memory manager. GPUs in the Task Manager We’re excited to introduce support for GPU performance data in the Task Manager. This is one of the features you have often requested,

devblogs.microsoft.com

 

[bsee-ino]

I'm running windows Server 2019, build 1809. This was also an issue in whatever version was before 1809.

 

[theguero]

I can confirm this issue. Installed a T4 onto my Supermicro X10DRLI-I motherboard today and the memory usage constantly shows 15360 MB usage.