Can i format to any allocation size?

[Artas1984]

I am confused. I never expected that after so many years this fundamental hard drive understanding problem would occour to me...

I've read a long time ago, that since 2009 newer hard drive models can be formatted into 4096 byte allocation sectors.

This Western Digital Gold series hard drive sheet shows even the newest hard drives still having ''only'' 4K physical bytes per sector.

Why and how then in Windows hard drive management You have the option to format hard drives into 64K sectors? If the sector size is limited by the physical specifications of the hard drive, how in the blue hell can you format above 4K? Or if the hard drive is limited to 512 bytes physical sectors, how can you format above 512 bytes?

I have a 4 TB Seagate Enterprise hard drive, which is rated at 4K allocation by default. I am using it for movies only, so i obviously and ideally i ''would like'' to format it to the biggest sector size possible..

 

[Flaky]

The allocation size refers to filesystem cluster size.

The disk's sector size is rather a low-level detail of how HDD/SSD operate, and one should differentiate between hardware sector size, and logical sector size. That's where 512n/512e/4Kn comes into play.

There's nothing wrong with having clusters that span over multiple sectors.

 

[Athlonite]

There's nothing wrong with having clusters that span over multiple sectors.

Except if you use 64K sector format you waste alot of space ie: you have a file that's 8K in size it's stored to your HDD/SSD in an 64K sector 64 - 8 = 56K that's 56K that can't be used because that's been designated as used by on little ole 8K file.. with 4k sectors you'd use 2 x 4K sectors to store the same file but loose no free space

 

[lexluthermiester]

Except if you use 64K sector format you waste alot of space ie: you have a file that's 8K in size it's stored to your HDD/SSD in an 64K sector 64 - 8 = 56K that's 56K that can't be used because that's been designated as used by on little ole 8K file.. with 4k sectors you'd use 2 x 4K sectors to store the same file but loose no free space

Correct, which is why 4k sectors are still being used.

@Artas1984 Use whatever your needs require. As long as the utility you're using has an option, it can be used. Windows will recognize and use whatever it is given as long as it fits within the established specs for the standard.
For example;
If you are going to use a large drive/partition(2TB+) primarily for large files(raw video, audio, images) than 64k filesystem sectors will fit your needs well and will not be storage space inefficient.
However, if you are creating a boot drive, regardless of size, you want to go with 4k sectors as there will be a great many small files installed and you want to use sector sizes that are going to be space efficient.

 

[eidairaman1]

I am confused. I never expected that after so many years this fundamental hard drive understanding problem would occour to me...

I've read a long time ago, that since 2009 newer hard drive models can be formatted into 4096 byte allocation sectors.

This Western Digital Gold series hard drive sheet shows even the newest hard drives still having ''only'' 4K physical bytes per sector.

Why and how then in Windows hard drive management You have the option to format hard drives into 64K sectors? If the sector size is limited by the physical specifications of the hard drive, how in the blue hell can you format above 4K? Or if the hard drive is limited to 512 bytes physical sectors, how can you format above 512 bytes?

I have a 4 TB Seagate Enterprise hard drive, which is rated at 4K allocation by default. I am using it for movies only, so i obviously and ideally i ''would like'' to format it to the biggest sector size possible..

I do recall that Windows XP after SP2 you couldnt use a 32K cluster, only a 4K cluster, or was that SP3 and Win7+...

https://en.m.wikipedia.org/wiki/Data_cluster

Smaller cluster sizes reduced wasted space where storage on a desktop is limited. Larger clusters were pretty fast but only useful for big files and where storage is virtually unlimited in servers/mainframes/supercomputers

 

[Artas1984]

Is there a program that would allow me to format my HDD to 1 MB allocation clusters? Windows default max is 64 KB.

 

[Shambles1980]

you want the smallest allacotion you can..

imagine you have a huge room.. (your hard disk) and you want to store things in there. so you buy boxes (clusters).. But you can only put 1 thing in every box (files)..
the boxes are like jigsaws that you can combine together so you can fit a really big object in it. Bu you can never make them smaller.
You will run out of boxes reall fast with lots of small items, but the room would never really be fulll, it would just look full..

OR..

a 1kb text file will take up 64kb of space on your hdd with a 64kb cluster. or 1mb of space with a 1mb cluster

Really basic explanation dont kill me here.

 

[Artas1984]

you want the smallest allacotion you can..

I want the max allocation i can. HDD will only store BIG SIZE single file movies, nothing else.

 

[Shambles1980]

Google command line formating variables. this should allow you to set a cluster size, although i havent done anything like that since dos, it should still be an option to specify via command line.

actually..

open cmd
format x: /fs:ntfs /a:yyyyy
should probably do it.. (yyyy is cluster size) x is hdd
but do format /? 1st to be sure

 

[lexluthermiester]

Google command line formating variables. this should allow you to set a cluster size, although i havent done anything like that since dos, it should still be an option to specify via command line.

Nah, he doesn't need that.

I want the max allocation i can. HDD will only store BIG SIZE single file movies, nothing else.

Get a partitioning utility like AOMEI or EaseUS;
https://www.majorgeeks.com/files/details/aomei_partition_assistant.html
https://www.majorgeeks.com/files/details/easeus_partition_master_home_edition.html

Both have the ability to format a partition to whatever maximum sector size the file system supports.

 

[Flaky]

Is there a program that would allow me to format my HDD to 1 MB allocation clusters? Windows default max is 64 KB.

That's interesting - in my case windows has no problem listing up to 2048KB cluster sizes. And that's everywhere - from format dialog, to diskpart.
But that's on windows 10.

I want the max allocation i can. HDD will only store BIG SIZE single file movies, nothing else.

What gains are you expecting, exactly? Is this based solely on assumption, or do you have any benchmarks showing noticeable benefits by going beyond 64KB?

 

[lexluthermiester]

What gains are you expecting, exactly? Is this based solely on assumption, or do you have any benchmarks showing noticeable benefits by going beyond 64KB?

There is a benefit. I've seen differences of up to 40% drive performance between 8k and 32k sector sizes. The larger the sector size, the less time the HDD heads need to spend writing to/reading from the partition table, thus dramatically cutting down on seek times as well. However, this has the downside of wasting space on small files, which the OP will not be storing or will have a minimum of. In his usage scenario, the wasted space will be very little and he will benefit from the larger sector sizes.

 

[Flaky]

Did you miss the "beyond 64KB" part in question?

 

[lexluthermiester]

Did you miss the "beyond 64KB" part in question?

I did not.

 

[SoNic67]

the less time the HDD heads need to spend writing to/reading from the partition table

That is cached in RAM in any modern OS, no need to be constantly read from disk.

 

[lexluthermiester]

That is cached in RAM in any modern OS, no need to be constantly read from disk.

But the drive still has to load the data into cache from the disc. That takes time, a lot of it if the system requests data that isn't in the cache.

 

[SoNic67]

When the RAM is sufficient, all the partition info will fit into Windows cache.
I would like to see HDD testing done with different sizes on a modern computer with 16GB of RAM (or more).
Now, keep in mind that hardware evolves, what was consensus 10 years ago might not be true anymore. Example: I am using a RAID5 with stripe size 64kB. 20 years ago that was crazy, standard was 256kB for RAID 1 and 5 wasn't even an option for performance reasons. I was using SCSI controllers and drives because IDE was a joke.

 

[lexluthermiester]

I am using a RAID5 with stripe size 64kB.

Ouch. 8k is the standard size set whenever I set up an array for someone if it's going to be bootable. If it's not bootable, then I default to 16k or 32k depending on the capacity of the array.

and 5 wasn't even an option for performance reasons.

Rubbish! I've been using RAID5 since the SCSI 50pin connector days. RAID5 offers performance boost AND data redundancy which is what made it so appealing.

I was using SCSI controllers and drives because IDE was a joke.

IDE RAID was a joke with low end software driven controllers. You get a hardware IDE RAID controller and the difference was night and day better.
We're a bit off topic here.

With mechanical drives, the PT & file system TOC being loaded into cache is only part of the equation. The data contained still needs to be accessed and that takes seek time based operations.

 

[SoNic67]

Ouch. 8k is the standard size set whenever I set up an array for someone if it's going to be bootable. If it's not bootable, then I default to 16k or 32k depending on the capacity of the array.

I am sorry for your clients. I don't know what controllers you have, but in my testing on 4 different controllers, going from 64kB to 16kB cuts in half the read performance and crushes the writes (less that 1/4). It's normal, the needed calculations are 4 times more intensive.

PS: My fastest card is an LSI 9750-8i, x8 PCIe 2.0, 6Gbps, 512MB cache, iBBU07 battery backup unit.

 

[lexluthermiester]

I am sorry for your clients. I don't know what controllers you have, but in my testing on 4 different controllers, going from 64kB to 16kB cuts in half the read performance and crushes the writes (less that 1/4). It's normal, the needed calculations are 4 times more intensive.

PS: My fastest card is an LSI 9750-8i, x8 PCIe 2.0, 6Gbps, 512MB cache, iBBU07 battery backup unit.

You say that like you think I haven't done side by side benchmark testing. I have and 8k performs better on a bootable array, for mechanical HDD's.

 

[SoNic67]

My Cristal tests are posted for last two controllers with 64k stripes.
I did testing with 16k initially and instead of sequential read 400, I got 250, instead of seq writes of 325, I got 60. Did I screenshoot those to "prove" anything? Nope. And I won't re-do my RAID ( a whole day) just to prove anything here.
It's just a well known fact that the number of stripes processed per second is limited by the computation power of the hardware engine. So for that identical ammount of stripes/second, smaller sized stripes results in smaller MB/s troughput.
The only disadvantage it just that bigger stripes might waste more space for small files.

But, since you are asking my proof, I am sure that YOU have all the proof that 16 and 8 are as great as 64...

 

[Shambles1980]

I think you guys scared away the op.

 

[lexluthermiester]

I think you guys scared away the op.

Nah, Artas1984 isn't scared away that easily..