• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

hard disk speed with file size question

W

wolf2009

Guest
Hard disk speed depends on the file size, right ?

Like if you have 1000 small files each 1MB, it would take longer to transfer them, than 1 file of 1000 MB.

My question is, Why so ?
 

Autiem Sel

New Member
Joined
Jul 26, 2007
Messages
23 (0.00/day)
Location
Glasgow
Processor intel e6700 @ 2.91ghz
Motherboard asrock 4coredual-vsta
Cooling 2x akasa 80mm, arctic cooling freezer 7 pro
Memory 2gb corsair xms2
Video Card(s) his x1950pro agp 512mb + accelero x2
Storage 120gb & 80gb ide
Display(s) 19" dell p991
Case found in street
Audio Device(s) onboard audio
Power Supply coolermaster igreen 500w
Software windaes xp home sp2
It would take longer to eat 1000 apples than eat 1 huge apple. Something to do with surface area I think.
 

AsRock

TPU addict
Joined
Jun 23, 2007
Messages
18,853 (3.08/day)
Location
UK\USA
Processor AMD 3900X \ AMD 7700X
Motherboard ASRock AM4 X570 Pro 4 \ ASUS X670Xe TUF
Cooling D15
Memory Patriot 2x16GB PVS432G320C6K \ G.Skill Flare X5 F5-6000J3238F 2x16GB
Video Card(s) eVga GTX1060 SSC \ XFX RX 6950XT RX-695XATBD9
Storage Sammy 860, MX500, Sabrent Rocket 4 Sammy Evo 980 \ 1xSabrent Rocket 4+, Sammy 2x990 Pro
Display(s) Samsung 1080P \ LG 43UN700
Case Fractal Design Pop Air 2x140mm fans from Torrent \ Fractal Design Torrent 2 SilverStone FHP141x2
Audio Device(s) Yamaha RX-V677 \ Yamaha CX-830+Yamaha MX-630 Infinity RS4000\Paradigm P Studio 20, Blue Yeti
Power Supply Seasonic Prime TX-750 \ Corsair RM1000X Shift
Mouse Steelseries Sensei wireless \ Steelseries Sensei wireless
Keyboard Logitech K120 \ Wooting Two HE
Benchmark Scores Meh benchmarks.
I think the 1GB file be faster as long as all files were defraged correctly.
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
26,957 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit
seeks will kill performance. another factor is OS overhead, "processing" a lot of small files takes more computing power than just one big chunk of data
 
Joined
Nov 10, 2008
Messages
1,982 (0.35/day)
Processor Intel Core i9 9900k @ 5.1GHZ all core load (8c 16t)
Motherboard MSI MEG Z390 ACE
Cooling Corsair H100i v2 240mm
Memory 32GB Corsair 3200mhz C16 (2x16GB)
Video Card(s) Powercolor RX 6900 XT Red Devil Ultimate (XTXH) @ 2.6ghz core, 2.1ghz mem
Storage 256GB WD Black NVME drive, 4TB across various SSDs/NVMEs, 4TB HDD
Display(s) Asus 32" PG32QUX (4k 144hz mini-LED backlit IPS with freesync & gsync & 1400 nit HDR)
Case Corsair 760T
Power Supply Corsair HX850i
Mouse Logitech G502 Lightspeed on powerplay mousemat
Keyboard Logitech G910
VR HMD Wireless Vive Pro & Valve knuckles
Software Windows 10 Pro
Due to hard drives physically being seperated into set block sizes where one block can only contain data for ONE file if you have 1000x 1MB files which do not use all the space in their blocks then they will need more blocks than 1x 1000MB file which would only have 1 block where all the space wouldn't be used as it would fill each block before moving onto the next one. So if each of the 1000 1MB blocks wastes 1 block then they'd need 999 more hard drive blocks than the 1000MB file.

Plus the added multiple file overhead and extra info needed for each seperate file and the fact that the 1000 smaller files even if defragged could be on completely different sectors of the hard drive platter, making the reading head having to jump all over the disk, whereas a single large file if properly defragmented should be in a sequential order, removing the seek time needed very significantly.
 

Mussels

Freshwater Moderator
Staff member
Joined
Oct 6, 2004
Messages
58,413 (8.21/day)
Location
Oystralia
System Name Rainbow Sparkles (Power efficient, <350W gaming load)
Processor Ryzen R7 5800x3D (Undervolted, 4.45GHz all core)
Motherboard Asus x570-F (BIOS Modded)
Cooling Alphacool Apex UV - Alphacool Eisblock XPX Aurora + EK Quantum ARGB 3090 w/ active backplate
Memory 2x32GB DDR4 3600 Corsair Vengeance RGB @3866 C18-22-22-22-42 TRFC704 (1.4V Hynix MJR - SoC 1.15V)
Video Card(s) Galax RTX 3090 SG 24GB: Underclocked to 1700Mhz 0.750v (375W down to 250W))
Storage 2TB WD SN850 NVME + 1TB Sasmsung 970 Pro NVME + 1TB Intel 6000P NVME USB 3.2
Display(s) Phillips 32 32M1N5800A (4k144), LG 32" (4K60) | Gigabyte G32QC (2k165) | Phillips 328m6fjrmb (2K144)
Case Fractal Design R6
Audio Device(s) Logitech G560 | Corsair Void pro RGB |Blue Yeti mic
Power Supply Fractal Ion+ 2 860W (Platinum) (This thing is God-tier. Silent and TINY)
Mouse Logitech G Pro wireless + Steelseries Prisma XL
Keyboard Razer Huntsman TE ( Sexy white keycaps)
VR HMD Oculus Rift S + Quest 2
Software Windows 11 pro x64 (Yes, it's genuinely a good OS) OpenRGB - ditch the branded bloatware!
Benchmark Scores Nyooom.
seeks will kill performance. another factor is OS overhead, "processing" a lot of small files takes more computing power than just one big chunk of data

its exactly what w1zz said. no surprise there.

Theres no guarantee your 1,000 small files are going to be located near each other on the hard drive - with a (very) rough average of 10ms seek times on hard drives, you could have a lot of seeks, wheras a single (defragmented) file could be done in one smooth transfer.

if you look at benchmark programs and so on, you'll often see "sustained" vs "random" in read and write tests. random would be your 1,000 small files, sustained one large file.
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
It's easier for a file system to look at a big file, segment it, and throw it on a drive than micro-manage 1000 individual files that may or may not be related. Moreover, you have 1000 times much information in the file system telling it what each individual file is.

Small files are less likely to fragment than larger files but many small files take longer to write (due to updates to the file system) than few large files.
 

btarunr

Editor & Senior Moderator
Staff member
Joined
Oct 9, 2007
Messages
46,283 (7.69/day)
Location
Hyderabad, India
System Name RBMK-1000
Processor AMD Ryzen 7 5700G
Motherboard ASUS ROG Strix B450-E Gaming
Cooling DeepCool Gammax L240 V2
Memory 2x 8GB G.Skill Sniper X
Video Card(s) Palit GeForce RTX 2080 SUPER GameRock
Storage Western Digital Black NVMe 512GB
Display(s) BenQ 1440p 60 Hz 27-inch
Case Corsair Carbide 100R
Audio Device(s) ASUS SupremeFX S1220A
Power Supply Cooler Master MWE Gold 650W
Mouse ASUS ROG Strix Impact
Keyboard Gamdias Hermes E2
Software Windows 11 Pro
The OS has to create file system index entries for each file ?
 
Joined
Apr 26, 2009
Messages
513 (0.09/day)
Location
You are here.
System Name Prometheus
Processor AMD Ryzen 9 5950x
Motherboard ASUS ROG Strix B550-I Gaming
Cooling EKWB EK-240 AIO D-RGB
Memory G.Skill Trident Z Neo 32GB
Video Card(s) MSI RTX 4070Ti Ventus 3X OC 12GB
Storage WD Black SN850 1TB + 1 x Samsung 970 Evo Plus 2TB
Display(s) DELL U4320Q 4K + Wacom Cintiq Pro 16 4K
Case Jonsbo A4 ver1.1 SFF
Audio Device(s) ASUS SupremeFX S1220A
Power Supply Corsair SF750 Platinum SFX
Mouse Logitech Pro Wireless
Keyboard Vortex Race 3 75% MX Brown
Software Windows 11 Pro x64
Move a box of m&m's from one location to another. Now move them back, but this time one candy at a time. Does it take longer?

Of course the box is heavy and you'll have difficulties moving it, so your movement will be partially impared. A candy is easier to move, but you'll have to repeat a few complex steps to move just one. And you have a whole bunch of them to move. So you'll repeat the same steps over and over again.

Unless you get hungry and eat a few, then the OS will give you an IO error.
 
Last edited:
Top