• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

exFAT is REALLY slow wrt entry operations

Joined
Feb 14, 2012
Messages
2,192 (0.58/day)
System Name msdos
Processor 8086
Motherboard mainboard
Cooling passive
Memory 640KB + 384KB extended
Video Card(s) EGA
Storage 5.25"
Display(s) 80x25
Case plastic
Audio Device(s) modchip
Power Supply 45 watts
Mouse serial
Keyboard yes
Software disk commander
Benchmark Scores still running
I discovered that even when the directory structure is cached in ram, exFAT is like 60x slower than NTFS, it's literally CPU bound just to traverse large source code directory structures. This is like when I figured out ext3/4 was so much faster than ReiserFS with loads of rpm files, it was the difference between instantaneous and 10 seconds just to iterate over the repo filenames. I think the data transfer rate (read/write) itself is fine. I can only guess that Windows is reparsing a lot of the exFAT table when looking up each entry. exFAT is about 80/second whereas NTFS is close to 5000. I get that NTFS (whatever Win11 uses these days) is their bread and butter filesystem, but that's obnoxiously slow for exFAT.
 
Joined
Aug 20, 2007
Messages
17,867 (3.29/day)
System Name Pioneer
Processor Ryzen R9 5950X
Motherboard GIGABYTE X570 Aorus Elite
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory Crucial Ballistix 64GB (4 x 16GB) @ DDR4-3600 (Micron E-Die, dual rank sticks)
Video Card(s) EVGA GeForce RTX 3090 Ti FTW3
Storage 2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply EVGA SuperNova T2 Titanium 850W
Mouse Razer Deathadder v2
Keyboard WASD CODE Mechanical KB w/ Cherry MX Green switches
Software Windows 11 Enterprise (yes, it's legit)
FAT is a horrible filesystem, and exFAT is just the 64-bit inode version of it so... it could just be it performs that bad due to the fact it's a shitty filesystem.
 
Joined
Feb 14, 2012
Messages
2,192 (0.58/day)
System Name msdos
Processor 8086
Motherboard mainboard
Cooling passive
Memory 640KB + 384KB extended
Video Card(s) EGA
Storage 5.25"
Display(s) 80x25
Case plastic
Audio Device(s) modchip
Power Supply 45 watts
Mouse serial
Keyboard yes
Software disk commander
Benchmark Scores still running
It's a very simple filesystem with the name and filesize enumerated in the directory entries. I am guessing that Windows might be validating filesize against tracing the FAT table, or something like that, every time. There's no technical reason for it to be so slow, as far as I can tell. I should check Linux out of curiosity.
 
Joined
Aug 20, 2007
Messages
17,867 (3.29/day)
System Name Pioneer
Processor Ryzen R9 5950X
Motherboard GIGABYTE X570 Aorus Elite
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory Crucial Ballistix 64GB (4 x 16GB) @ DDR4-3600 (Micron E-Die, dual rank sticks)
Video Card(s) EVGA GeForce RTX 3090 Ti FTW3
Storage 2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply EVGA SuperNova T2 Titanium 850W
Mouse Razer Deathadder v2
Keyboard WASD CODE Mechanical KB w/ Cherry MX Green switches
Software Windows 11 Enterprise (yes, it's legit)
I suppose the slowness would be more on HDDs than flash given we now have good access times... yeah good point.

Also, being unjournaled filesystem checks are kinda poopy, but that's totally unrelated.
 

qubit

Overclocked quantum bit
Joined
Dec 6, 2007
Messages
17,316 (3.25/day)
Location
Quantum Well UK
System Name Quantumville™
Processor Intel Core i7-2700K @ 4GHz
Motherboard Asus P8Z68-V PRO/GEN3
Cooling Noctua NH-D14
Memory 16GB (2 x 8GB Corsair Vengeance Black DDR3 PC3-12800 C9 1600MHz)
Video Card(s) MSI RTX 2080 SUPER Gaming X Trio
Storage Samsung 850 Pro 256GB | WD Black 4TB | WD Blue 6TB
Display(s) BenQ XL2720Z (144Hz, 3D Vision 2, 1080p) | Asus MG28UQ (4K, 60Hz, FreeSync compatible)
Case Cooler Master HAF 922
Audio Device(s) Creative Sound Blaster X-Fi Fatal1ty PCIe
Power Supply Corsair HX 850W v1
Mouse Microsoft Intellimouse Pro - Black Shadow
Keyboard Yes
Software Windows 10 Pro 64-bit
FAT is a horrible filesystem, and exFAT is just the 64-bit inode version of it so... it could just be it performs that bad due to the fact it's a shitty filesystem.
I agree it's likely just badly written. There's another downside to this: potentially poorer file integrity. Think about it, files are lost at risk when being written and the filenames / directory structure being updated, so one wants to get these operations over with as quickly as possible. Stretching it out like this is just asking for trouble.
 
Joined
Feb 14, 2012
Messages
2,192 (0.58/day)
System Name msdos
Processor 8086
Motherboard mainboard
Cooling passive
Memory 640KB + 384KB extended
Video Card(s) EGA
Storage 5.25"
Display(s) 80x25
Case plastic
Audio Device(s) modchip
Power Supply 45 watts
Mouse serial
Keyboard yes
Software disk commander
Benchmark Scores still running
I used to use a FAT partition to dual-boot between Windows and Linux to share files. But FAT's file timestamp resolution is only 2 seconds (!), whereas exFAT is 10ms (wrt source code and object file time stamps). 10ms is still bad though (ext4 is 1ns, ntfs is 0.1ns).

Maybe you missed my point about exFAT being slow -- I pointed out it was still slow when no actual device access was occurring, so 100% cached in ram. It's whatever algorithm they are using is very unoptimized for traversing directory entries.
 
Joined
Aug 20, 2007
Messages
17,867 (3.29/day)
System Name Pioneer
Processor Ryzen R9 5950X
Motherboard GIGABYTE X570 Aorus Elite
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory Crucial Ballistix 64GB (4 x 16GB) @ DDR4-3600 (Micron E-Die, dual rank sticks)
Video Card(s) EVGA GeForce RTX 3090 Ti FTW3
Storage 2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply EVGA SuperNova T2 Titanium 850W
Mouse Razer Deathadder v2
Keyboard WASD CODE Mechanical KB w/ Cherry MX Green switches
Software Windows 11 Enterprise (yes, it's legit)
It's whatever algorithm they are using is very unoptimized for traversing directory entries.
I think that's because it's coding priority filesystem wise is very low. Microsoft has been kind of wishing FAT would just go away for years, doubtful they put good coders on it.
 
Top