• We've upgraded our forums. Please post any issues/requests in this thread.

hard disk speed with file size question

W

wolf2009

Guest
#1
Hard disk speed depends on the file size, right ?

Like if you have 1000 small files each 1MB, it would take longer to transfer them, than 1 file of 1000 MB.

My question is, Why so ?
 

Autiem Sel

New Member
Joined
Jul 26, 2007
Messages
23 (0.01/day)
Likes
0
Location
Glasgow
Processor intel e6700 @ 2.91ghz
Motherboard asrock 4coredual-vsta
Cooling 2x akasa 80mm, arctic cooling freezer 7 pro
Memory 2gb corsair xms2
Video Card(s) his x1950pro agp 512mb + accelero x2
Storage 120gb & 80gb ide
Display(s) 19" dell p991
Case found in street
Audio Device(s) onboard audio
Power Supply coolermaster igreen 500w
Software windaes xp home sp2
#2
It would take longer to eat 1000 apples than eat 1 huge apple. Something to do with surface area I think.
 

AsRock

TPU addict
Joined
Jun 23, 2007
Messages
15,320 (4.00/day)
Likes
4,726
Location
US
Processor 2500k \ 3770k
Motherboard ASRock Z68 \ Z77
Memory Samsung low profile 1600
Video Card(s) XFX 6770 \ XFX R9 290X
Storage Intel 80Gb (SATA2) WD 250Gb \ Team SSD+Samsung Evo 250Gb+500Gb+ 2xCorsair Force+WD250GbHDD
Display(s) Samsung 1080P \ Toshiba HDTV 1080P
Case HTPC400 \ Thermaltake Armor case ( original ), With Zalman fan controller ( wattage usage ).
Audio Device(s) Yamaha RX-V475 \ Marantz SR5008 Tannoy Mercury MKII Paradigm 5SE + Tannoy Mercury F4
Power Supply PC&Power 750w \ Seasonic 750w MKII
Mouse MS intelimouse \ Logitech G700s + Steelseries Sensei wireless
Keyboard Logitech K120 \ ROCCAT MK Pro ( modded amber leds )
Benchmark Scores Meh benchmarks.
#3
I think the 1GB file be faster as long as all files were defraged correctly.
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,072 (3.44/day)
Likes
17,990
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#4
seeks will kill performance. another factor is OS overhead, "processing" a lot of small files takes more computing power than just one big chunk of data
 
Joined
Nov 10, 2008
Messages
1,897 (0.57/day)
Likes
676
System Name Star Citizen Enabler
Processor Intel Core i7 4770K @ 4.2GHz
Motherboard MSI Z87-GD65
Cooling Triple 120 rad WC with EK-Supreme LTX CSQ waterblock
Memory 16GB Corsair Vengeance Pro 1866MHz C9
Video Card(s) MSI Gaming 6G GTX 980Ti
Storage Corsair Force 3 120GB, Kingstone V300 240GB, Seagate Barracuda 1.5tb, Samsung Spinpoint F1 1tb
Display(s) Oculus Rift DK2 + 24" 1920x1200 HP LP2475w(H-IPS panel)
Case Corsair Obsidian 800D
Audio Device(s) Asus Xonar D2X (pci-e) with optical into logitech z-5500 speakers
Power Supply Corsair HX850i
Software Windows 7 Professional 64-bit [full retail]
#5
Due to hard drives physically being seperated into set block sizes where one block can only contain data for ONE file if you have 1000x 1MB files which do not use all the space in their blocks then they will need more blocks than 1x 1000MB file which would only have 1 block where all the space wouldn't be used as it would fill each block before moving onto the next one. So if each of the 1000 1MB blocks wastes 1 block then they'd need 999 more hard drive blocks than the 1000MB file.

Plus the added multiple file overhead and extra info needed for each seperate file and the fact that the 1000 smaller files even if defragged could be on completely different sectors of the hard drive platter, making the reading head having to jump all over the disk, whereas a single large file if properly defragmented should be in a sequential order, removing the seek time needed very significantly.
 

Mussels

Moderprator
Staff member
Joined
Oct 6, 2004
Messages
46,130 (9.57/day)
Likes
13,564
Location
Australalalalalaia.
System Name Daddy Long Legs
Processor Ryzen R7 1700, 3.9GHz 1.375v
Motherboard MSI X370 Gaming PRO carbon
Cooling Fractal Celsius S24 (Silent fans, meh pump)
Memory 16GB 2133 generic @ 2800
Video Card(s) MSI GTX 1080 Gaming X (BIOS modded to Gaming Z - faster and solved black screen bugs!)
Storage 1TB Intel SSD Pro 6000p (60TB USB3 storage)
Display(s) Samsung 4K 40" HDTV (UA40KU6000WXXY) / 27" Qnix 2K 110Hz
Case Fractal Design R5. So much room, so quiet...
Audio Device(s) Pioneer VSX-519V + Yamaha YHT-270 / sennheiser HD595/518 + bob marley zion's
Power Supply Corsair HX 750i (Platinum, fan off til 300W)
Mouse Logitech G403 + KKmoon desk-sized mousepad
Keyboard Corsair K65 Rapidfire
Software Windows 10 pro x64 (all systems)
Benchmark Scores Laptops: i7-4510U + 840M 2GB (touchscreen) 275GB SSD + 16GB i7-2630QM + GT 540M + 8GB
#6
seeks will kill performance. another factor is OS overhead, "processing" a lot of small files takes more computing power than just one big chunk of data
its exactly what w1zz said. no surprise there.

Theres no guarantee your 1,000 small files are going to be located near each other on the hard drive - with a (very) rough average of 10ms seek times on hard drives, you could have a lot of seeks, wheras a single (defragmented) file could be done in one smooth transfer.

if you look at benchmark programs and so on, you'll often see "sustained" vs "random" in read and write tests. random would be your 1,000 small files, sustained one large file.
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
20,927 (6.24/day)
Likes
10,026
Location
IA, USA
System Name BY-2015
Processor Intel Core i7-6700K (4 x 4.00 GHz) w/ HT and Turbo on
Motherboard MSI Z170A GAMING M7
Cooling Scythe Kotetsu
Memory 2 x Kingston HyperX DDR4-2133 8 GiB
Video Card(s) PowerColor PCS+ 390 8 GiB DVI + HDMI
Storage Crucial MX300 275 GB, Seagate 6 TB 7200 RPM
Display(s) Samsung SyncMaster T240 24" LCD (1920x1200 HDMI) + Samsung SyncMaster 906BW 19" LCD (1440x900 DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay
Audio Device(s) Realtek Onboard, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse SteelSeries Sensei RAW
Keyboard Tesoro Excalibur
Software Windows 10 Pro 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
#7
It's easier for a file system to look at a big file, segment it, and throw it on a drive than micro-manage 1000 individual files that may or may not be related. Moreover, you have 1000 times much information in the file system telling it what each individual file is.

Small files are less likely to fragment than larger files but many small files take longer to write (due to updates to the file system) than few large files.
 

btarunr

Editor & Senior Moderator
Staff member
Joined
Oct 9, 2007
Messages
34,335 (9.22/day)
Likes
17,427
Location
Hyderabad, India
System Name Long shelf-life potato
Processor Intel Core i7-4770K
Motherboard ASUS Z97-A
Cooling Xigmatek Aegir CPU Cooler
Memory 16GB Kingston HyperX Beast DDR3-1866
Video Card(s) 2x GeForce GTX 970 SLI
Storage ADATA SU800 512GB
Display(s) Samsung U28D590D 28-inch 4K
Case Cooler Master CM690 Window
Audio Device(s) Creative Sound Blaster Recon3D PCIe
Power Supply Corsair HX850W
Mouse Razer Abyssus 2014
Keyboard Microsoft Sidewinder X4
Software Windows 10 Pro Creators Update
#8
The OS has to create file system index entries for each file ?
 
Joined
Apr 26, 2009
Messages
414 (0.13/day)
Likes
95
Location
You are here.
System Name Prometheus
Processor Intel i7 4930K
Motherboard Asus P9X79
Cooling Noctua NH-D14
Memory Crucial Ballistix Quad Channel 64GB DDR3 1.35V
Video Card(s) Asus GTX 970 4GB
Storage 2 x Intel 330 240GB RAID0 + 2 x Intel X-25M G2 160GB MIRROR + 4 x 2TB WD Green
Display(s) AMH A409U 4K
Case Intertech AP1
Audio Device(s) Creative Sound Blaster X-Fi Xtreme Audio PCIe
Power Supply Seasonic X660
Mouse Razer Death Adder 2013
Keyboard FILCO Majestouch 2 Ninja
Software Microsoft Windows 10 Pro x64
#9
Move a box of m&m's from one location to another. Now move them back, but this time one candy at a time. Does it take longer?

Of course the box is heavy and you'll have difficulties moving it, so your movement will be partially impared. A candy is easier to move, but you'll have to repeat a few complex steps to move just one. And you have a whole bunch of them to move. So you'll repeat the same steps over and over again.

Unless you get hungry and eat a few, then the OS will give you an IO error.
 
Last edited: