Slow USB transfers when CPU is idle

[The Von Matrices]

I've been struggling to fix an issue with slow USB transfers on a Debian 11 NAS, and I'm wondering if anyone has any suggestions. I have four external USB 3 HDDs connected to an Intel NUC6i3SYH. When I try to copy files over the network to the disks, the transfer only runs at 15-20 mbps even though the host server is capable of 1 gbps+. I look at CPU usage, I/O usage, and memory usage, and they are all at 10-20%, so none of them are the limitation.

Today, I stumbled across an odd finding. If I run any program that stresses at least one CPU core, the transfers speed up to 500mbps and the transfers are finally limited by the disks with I/O usage at 100%. Once that stressing processes stops, transfers slow down again back to 15-20mbps. I changed the CPU governor from "powersave" to "performance", and there was no noticeable change in transfer rates. It's only when I stress a CPU core that transfer rates speed up.

I'm thinking this is some sort of USB power saving feature taking effect, but I am not sure what config option to look for. Has anyone experienced this before or have a solution? Thanks!

 

[oobymach]

Is write caching enabled on the nas?

 

[DoLlyBirD]

Have you tried editing the power plan and making sure all power savings are off for USB, PCIE, CPU etc

 

[The Von Matrices]

Have you tried editing the power plan and making sure all power savings are off for USB, PCIE, CPU etc

This is running Debian 11, so there is not a "power plan" like Windows. Maybe there's something in the BIOS, but I'm not sure what to look for.

 

[Easy Rhino]

I've been struggling to fix an issue with slow USB transfers on a Debian 11 NAS, and I'm wondering if anyone has any suggestions. I have four external USB 3 HDDs connected to an Intel NUC6i3SYH. When I try to copy files over the network to the disks, the transfer only runs at 15-20 mbps even though the host server is capable of 1 gbps+. I look at CPU usage, I/O usage, and memory usage, and they are all at 10-20%, so none of them are the limitation.

Today, I stumbled across an odd finding. If I run any program that stresses at least one CPU core, the transfers speed up to 500mbps and the transfers are finally limited by the disks with I/O usage at 100%. Once that stressing processes stops, transfers slow down again back to 15-20mbps. I changed the CPU governor from "powersave" to "performance", and there was no noticeable change in transfer rates. It's only when I stress a CPU core that transfer rates speed up.

I'm thinking this is some sort of USB power saving feature taking effect, but I am not sure what config option to look for. Has anyone experienced this before or have a solution? Thanks!

Did you figure this out? It seems like a bug with the kernel drivers. There may be a patch for it.

 

[SchumannFrequency]

What type of HDD do you have connected via USB 3? Often the speeds indicated in Linux are not 100% accurate.
I'd be surprised if you're going to drive at 500MB/s over USB 3, unless you have a top model of an HDD connected.
Debian has some major problems right now: https://unixsheikh.com/articles/the-delusions-of-debian.html

I would currently recommend TrueNAS 13.0 which is seen as the best system for NAS systems:

TrueNAS 13.0-U1 Delivers Improved Performance

Experience enhanced performance and reliability with TrueNAS 13.0-U1 update. Discover improved scalability and features. Upgrade today!

www.truenas.com

FreeBSD simply achieves 2.5 times more IOPS than Linux currently does in ZFS, and Linux is also less stable for ZFS.
You can also format all your USB sticks in ZFS. I do this and both FreeBSD and Ubuntu automatically mount these USB sticks which is very useful.

Linux currently does not have a file system that is competitive with ZFS.
Btrfs on Linux is usually slower than ZFS on FreeBSD and Btrfs lacks certain useful features and also has a lot of bugs:

Examining btrfs, Linux’s perpetually half-finished filesystem

This btrfs filesystem overview highlights some longstanding shortcomings.

arstechnica.com

 

[The Von Matrices]

Just to follow up, I eventually fixed the issue.

I experimented with the different power saving settings in the BIOS. The PC is an Intel NUC6i3SYK. I eventually found that disabling PCIe link state power management in the BIOS fixed the problem. I have no idea why this fixed the issue considering that the USB ports are part of the chipset, but it is reproducible. The PC probably uses a bit more power now but it's a lot more usable as a NAS.