+80TB NAS

[AhokZYashA]

for NAS,
you can buy used xeon E3v4/v5 processors that supports ECC RAM,
some supported motherboard, some ECC RAM,

for SATA ports, i can suggest you get some of these
https://www.newegg.com/Product/Prod...re=sata_expansion_card-_-15-158-365-_-Product

its a expansion card with 4SATA ports, that plugs in to your PCI-e slots.

or you can find used LSI HBAs on ebay for quite cheap

 

[remi]

Personally OP should do more research on what software his FS gonna run on.

I looked at :
FreeNas
unRAID
Storage spaces

And the best one for my case is Storage spaces, because :
- there is no need for RAID hardware
- easily increase the size of the array
- you can add different types of drives

About the ECC RAM i'm not sure i need it and it would be to expensive, right ?

And if i buy a MoBo with 10 SATA ports then i dont need a
4 Port PCI Express 2.0 SATA III 6Gbps RAID Controller Card, which i think isnt recommended in Storage spaces

for NAS,
for SATA ports, i can suggest you get some of these
https://www.newegg.com/Product/Prod...re=sata_expansion_card-_-15-158-365-_-Product

its a expansion card with 4SATA ports, that plugs in to your PCI-e slots.

it says it's a RAID controler, would Storage space work if i connect my hdds through it ?

 

[ERazer]

ECC is always recommended if you have the budget for it and it is a tad expensive.

Depends on the OS for example on unRAID/FreeNas i use this raid card that has been flash to run IT mode to run as SATA expansion. I would check Storage Space forums.

 

[Static~Charge]

About the ECC RAM i'm not sure i need it and it would be to expensive, right ?

If you get a parity error in RAM, it could corrupt your data and/or your storage configuration. So, whether you choose to get ECC RAM depends on how highly you value your data on the file server.

it says it's a RAID controler, would Storage space work if i connect my hdds through it ?

If you look at the card's specifications, you'll see that it also supports JBOD (Just a Bunch Of Disks, meaning no RAID).

 

[remi]

ofcourse i value my data, so then can i use ECC RAM on ASRock X99 Extreme4 ?
And what cpu do i need for that ? xeon is out of my budget

What power supply should i get for 10 or more drives ?

 

[Steevo]

My 640 card supports port multipliers up to 20 devices out of the 4 ports.

 

[Static~Charge]

ofcourse i value my data, so then can i use ECC RAM on ASRock X99 Extreme4 ?

And what cpu do i need for that ? xeon is out of my budget

Did you look at the specifications for the ASRock X99 Extreme4?

Memory
- Supports DDR4 ECC, un-buffered memory/RDIMM with Intel Xeon processors E5 series in the LGA 2011-3 Socket

Most Intel desktop processors don't support ECC memory. You'll have to go with a Xeon to get that feature (i.e., no Celeron, Pentium, or Core i3 options).

Also, please note that the ASRock X99 Extreme4 is a Socket LGA 2011-3. This will limit your choice of processors. The least expensive one that I found in the current processor line-up was a Xeon E5-2603 V4, 1.7 GHz, 6 cores/6 threads, for $220. If you look through the motherboard's supported CPU list, you should be able to find a compatible, older processor for less money.

 

[Toothless]

I don't think OP realizes that this project is going to cost a good amount of money yet.

 

[remi]

I don't think OP realizes that this project is going to cost a good amount of money yet.

I agree )

But hey, a guy can dream right ?

Static~Charge thanks so much for that info. $220 for a cpu is a bit more then i wanted to spend, but at least it's not 1000$.

What if my RAM fails while it's creating the Parity for a new added drive ? Do i lose all my data ? Or with Storage Spaces you only lose one drive ?

btw, this database is what i need the NAS for :

 

[FordGT90Concept]

I'd recommend RocketRAID 840a but beware that it is not UEFI bootable. Put OS on an M.2 SSD. You'll need to buy 4x miniSAS -> 4x SATA breakout cables for it. It retails for $300 and the breakout cables can run for $30 each so you're looking at $420 just for RAID + cables. You're really not going to find a cheaper card than that to handle 16 drives.


Edit: Nanoxia DEEP SILENCE 6 only has room for 10 drives, 13 if some are converted (definitely don't recommend doing that because they're more likely to fail from heat).

Does this CPU need to transcode?

 

[remi]

I'd recommend RocketRAID 840a but beware that it is not UEFI bootable. Put OS on an M.2 SSD. You'll need to buy 4x miniSAS -> 4x SATA breakout cables for it. It retails for $300 and the breakout cables can run for $30 each so you're looking at $420 just for RAID + cables. You're really not going to find a cheaper card than that to handle 16 drives.

I dont want RAID, if the controler fails i lose ALL my data.
I will use Storage Spaces

Edit: Nanoxia DEEP SILENCE 6 only has room for 10 drives, 13 if some are converted (definitely don't recommend doing that because they're more likely to fail from heat).

The case can have up to 21 HDD and it has tones of fans so overheating wont be a problem.

 

[FordGT90Concept]

You don't *have* to use RAID. You can format the 16 drives individually if you so choose. That said, RAID cards tend to outlive their usefulness (got a 10 year old RocketRAID 3400 in my server). Should they fail, just buy another Highpoint card and it should detect the existing configuration.

Storage Spaces is software RAID. RAID cards (like the 840a) have a dedicated processor and RAM to perform all RAID parity processing in realtime without talking to the CPU. The RAID card monitors and recovers ECC issues and automatically ejects bad drives from the array.


My server has two Rexus Panaflo fans blowing directly across the HDDs and their oriented perpendicular to the case so it intakes from one side and exhausts out the other (not adding that heat to the CPU compartment).

 

[remi]

You're saying that if the RAID controler fails i wont lose the data ?

 

[FordGT90Concept]

The data is still on the drives. It just needs another Highpoint card to read that data and it will be like it never happened. Same thing applies to Intel software RAID: as long as the drives are plugged into another Intel chipset, it will detect the array like it was always there.

Ehm, when you create a RAID, the first thing it writes to the drives is the RAID type, the ID of the RAID, and the serial numbers of its members. So long as the RAID card knows how to read that data, it will use it. A Highpoint RAID won't be observed on an Adaptec or Intel chipset but it will be observed by other Highpoint cards.

Highpoint is cheap for what you get (Adaptec is twice as expensive, for example). And remember, it's an investment (almost 1mhr MTBF). You can easily move the PCIE card from one computer to another and, once the drivers are installed, it's all still there. You could even switch from Windows to Linux. Can't do that with Storage Spaces.


Are you intending to transcode on this system? That will dictate the rest of the system requirements. If not, you could get away with an Intel Atom, as long as the motherboard has an x8 slot for the RocketRAID card.

But yeah, not gonna happen for <$800. I think the minimum (assuming no transcoding) is going to be about $1000.

 

[remi]

For now

Does this CPU need to transcode?

No, i just need to have all my data in one place, nothing complicated. That's why i want to keep things as simple as possible with Storage Spaces.

 

[FordGT90Concept]

Here ya go:
https://www.newegg.com/Product/ProductList.aspx?Description=intel atom&Submit=ENE&IsNodeId=1&N=100007629 600456442

Big slot for RocketRAID, 4x10/100/1000 (82574L GbE), integrated graphics (Aspeed AST2400/AST2500 Graphic Driver), and an Atom processor (just be careful not to overload it on boot). The more expensive one is an octo-core. All you need is some DDR3 sticks, and a SATA SSD for the OS and you're golden.

Edit: Crap, PCIe is Gen 2, not Gen 3. A bit of a bottleneck but if you're plugging HDDs into the 840A, it should be fine.

No Linux drivers. Even supports ECC DDR3 because it's a server-grade Atom.

 

[remi]

What i dont get is, why do i need ECC RAM ? Cant i use 2 sticks of 4GB DDR4 NON-ECC and if one fails while parity is running i have the other one ?

NOTE that this is not going to be a true NAS, more like local data storage with 1 or 2 drive Parity.

For ex this guy isnt useing ECC or server grade cpu :

plus the MoBo : ASRock X99 Extreme4 doesnt even support ECC RAM

 

[FordGT90Concept]

You really don't need ECC memory. Just saying you can if you want to.

ECC is basically an insurance policy against radical electrons.

If you're getting a processor that supports it and it only costs a few bucks more to get ECC, why not?


Edit: Did some more digging on that SuperMicro Atom...
https://ark.intel.com/products/77983/Intel-Atom-Processor-C2558-2M-Cache-2_40-GHz

The 4xGbE is integrated into the CPU...can also function as 2x2.5GbE.

They actually make 16 core Atoms with 4x10GbE ports.
https://ark.intel.com/products/97927/Intel-Atom-Processor-C3958-16M-Cache-up-to-2_0-GHz

Review of that little beasty (no x8 so won't work for 840a):
https://www.servethehome.com/superm...ore-intel-atom-c3955-mitx-motherboard-review/

iKVM looks interesting.

 

[remi]

That's a releif, cause this media server will definetly not be operational 24/7. More like a few hours a day.

But in worst case scenario, can i lose 80TB of data just because i didnt opt for the ECC RAM ?

FordGT90Concept, SuperMicro Atom seems ok (to few sata 3 ports though) but i'm not in the US and i cant find it in Romania

 

[FordGT90Concept]

Just look for motherboards with Atom C-series processors and you should be on the right path.

In all honesty, any motherboard with enough network bandwidth and a PCIE x8 slot will work if you have an 840a. Remember, 1 GbE ~= 110 MB/s. Your drives each will support 200+ MB/s. For that reason, dual and even quad 1 GbE looks very attractive to me.

That's a releif, cause this media server will definetly not be operational 24/7. More like a few hours a day.

But in worst case scenario, can i lose 80TB of data just because i didnt opt for the ECC RAM ?

I seriously wouldn't worry about it. If you want to learn more about ECC, here you go:
https://www.pugetsystems.com/labs/articles/Advantages-of-ECC-Memory-520/

The best way to prevent ECC-like errors is to use RAID with write-through caching enabled. Data is verified as it is being written and, through parity information, confirmed (and fixed if necessary) as it is being read. Too many errors on a specific drive and it will throw it out of the RAID. 840a supports this and if you have their RAID management software running, it can even send you an email that specific serial # has been removed and needs to be replaced.

For the record, you can use Storage Spaces on top of the hardware RAID. I'm sure Microsoft Azure does this.

 

[londiste]

You need to figure out the solution you are going with as well as any additional requirements.

Just some thougths:
- Whether it's RAID, unRAID, FreeNAS, Storage Spaces or something else, the idea behind making sure data is not corrupt or lost remains the same. There must be some extra space for clone and/or parity information. Which particular implementation you choose is fairly irrelevant here. Recovery process may become equally tricky in each of them if something crucial is lost.
- Hardware(-assisted) RAID vs software-based solutions usually becomes a question of performance. Software-based solutions will be heavier on CPU and possibly memory when you need good throughput/IOPS or whichever performance measure. In your case - for archiving - you probably will not have to worry about that aspect.
- RAM does not run in pairs, parity is exactly what ECC is for. ECC is an insurance policy against any memory errors. ZFS is claimed to be particularly vulnerable due to heavy reliance on RAM but most software solutions will have some level of RAM caching.
- Have you looked into requirements for Storage Spaces - CPU, RAM, operating system (licensing)? For what you need and with what you have in mind for budget, a free solution with less complecity and lower needs from hardware is probably more than sufficient.
- For simple storage and archiving, the hardware does not have to be particularly fast when it comes to CPU, RAM etc. Power, cooling and reliability is what to look for.
- Do you have any needs or wishes towards the case itself - pretty, small, large, quiet, convenient, rackable, resistant to dust/temperature?

With what you have written about your needs and budget, I would probably go for FreeNAS or one if its analogues on an Atom-based server/NAS oriented boards. Something like this:
http://www.supermicro.com/products/motherboard/atom/A2SDi-2C-HLN4F.cfm
http://www.supermicro.com/products/motherboard/atom/A2SDi-8C-HLN4F.cfm
(not 100% sure on their prices but lower end of the range should be within budget).

I have a weakness towards cases that I could stash somewhere in living space and would look good enough. For this kind of purpose, I particularly like this one: http://www.lian-li.com/en/dt_portfolio/pc-q26/
But that is very subjective suggestion on my part and not a cheap one

Larger NAS appliances should also be within your budget. A couple Synology's 8-disk NASes should be $800-900 if I remember correctly. I am sure their competitors have some as well. These are extremely worry-free when compared to custom-built NAS, unless you enjoy tinkering and learning about technical aspects of a NAS.

 

[FordGT90Concept]

Keep in mind that 10+ TB drives will be using at least 512e format. May want to consider 4Kn:
https://en.wikipedia.org/wiki/Advanced_Format

http://www.seagate.com/enterprise-storage/hard-disk-drives/enterprise-capacity-3-5-hdd-helium/
http://www.hgst.com/products/hard-drives/ultrastar-he10 (4th from last digit in model number: N = 4Kn, E = 512e)

That's going to limit your options in terms of hardware support. Also operating system (e.g. Windows 7 doesn't support it).

 

[remi]

didnt know that, atm on win 7

Honestly, every person i talk to recommneds another thing.
RAID, unRAID, FreeNAS or Storage spaces ?

I found out that there are many cases where Storage spaces got a bug, and people lost all their data.
unRAID is for Linux and i do prefer Windows.
RAID is the most complicated/expensive, so pass...
FreeNAS requires tones of RAM and you can't add different size drives, so pass...

londiste - for the case i do love the Nanoxia DEEP SILENCE 6, it can hold 21 HDDs !!!

 

[FordGT90Concept]

RAID is not complicated. Seriously, it only has five settings when creating the RAID and it's transparent after that.

1) RAID type:
RAID5 = minimum 3 drives, 1 can fail
RAID50 = RAID5 + RAID5 = minimum 6 drives, one can fail in each RAID simultaneously without data loss
RAID6 = minimum 4 drives, 2 can fail

RAID6 and 50 you would need 10 x 10 TB to get 80 TB
RAID5 you would need 9 x 10 TB to get 80 TB

2) Volume Name: e.g. "Storage"

3) Sector Size: 512 (512e drives) or 4096 (4Kn drives).
512 ends at 6TB drives, hence change to 512e (e as in emulated, 4K underneath).

4) Cache Method:
Write-Thru: confirms write after the data hit the platters.
Write-Back: confirms write when the data is in the RAID-card's RAM (not yet written but queued to write).

5) Creation Method:
Foreground: it creates the RAID array right now. It can't be used until it is done.
Background: it creates the RAID array in the background. Performance is slow until it is done but it will accept read/write requests.


Once you get into the operating system, install the drivers for the card and it you'll see the volume as unformatted space. Create a GPT partition on it, assign it a letter, and away you go.


FreeNAS, if I understand it correctly, is like Active Directory on Windows Server.

 

[londiste]

remi, you definitely need to figure out what the end result should be.
Adding a new hard drive every month does not seem like something you would want to deal with when it comes to any kind of storage array.

How large is the amount of data you intend to store eventually?
How do you intend to access the data, how often?
What do you need in terms of backup/redundancy?
Is this the only copy of data or do you intend to have a backup of entire thing?

Do you even need it to be NAS? You have mentioned only using it occasionally and over USB3, NAS by definition is accessed over network and normally always online.
Do you even need a storage array? Perhaps an archive? As the simplest example - a library of hard disks and USB dock?

FreeNAS, if I understand it correctly, is like Active Directory on Windows Server.

I believe that part of Windows Server with functionality similar to FreeNAS is called File and Storage Services
Linux can be made to do a lot of things, including a LDAP directory but that's not really built into FreeNAS nor the purpose of it.