Large NAS server with backup - Advise or input welcome

[Jhelms]

So here is where I am at. For my HT running kodi and also general data, I am looking into building a high capacity NAS setup. Admittedly, this is rather new territory for me. I am used to just having several backup drives and a 2TB drive on my routers built in NAS.

Overall for the amount of data, I would need 12TB with 12TB of backup for redundancy at minimum. Possibly 16-18TB optimal.

I currently see 4TB drives being the best bang for the buck with WD greens at $140. I have no issue doing 6tb drives, however I do not see any NAS setups that offer 6TB support. Or should I just skip a dedicated NAS and build my own?

My raid experience comes down to just raid 1 and 0. Seems what I am after would be a bit more complex than the basic two drive setups I have done in the past (researching other raid options currently)

Any info from others that have built a similar setup would be appreciated.

 

[Kursah]

Look into Raid5 or 6, for redundancy. Raid 5 requires 3+ drives and you get around 70% of your combined space, the rest is used for redundancy. What this means is if 1 drive fails, you replace it, and the array can rebuild.

Raid6 expands on this, but drops closer to 60% iirc (these percentages are rough figures), but allows for 2 drive failures on a 4+ drive array iirc and the array is still rebuildable.

If you want redundancy, you could do a nested raid, say a Raid 10 (1+0), or similar as well: http://en.wikipedia.org/wiki/Nested_RAID_levels#RAID_10

Depends on your needs.

Needing that much space, I would consider mirroring your main array. But then my experience with that much data is limited to doing just that though, so my suggestion is slightly biased.

Hope that helps a little! Spend some time researching and you'll get some other good info here I'm sure, and then be able to make the best decision for you and your network.

 

[rtwjunkie]

My only comment would be to reconsider your HDD choice for storage. I, like a ton of other users, have had high failure rates on the WD Greens in my server. Once I went to WD Reds (and a couple Blacks), I have had a no failure rate.

 

[Brusfantomet]

I am using a home built server with 4 x 4 TB drives in RAID 6, I did that because after comparing the prices I fund that a 8 bay NAS server would cost much more than what a basic computer with a ok raid card would cost.


Use Free nas or some linux OS of your choice, I am using windows server 2008 because I had some free copies from my university days.


I am using a AMD APU but a intel i3 will also be more than good enough.

For RAID card I use a High Point rocket RAID 2720 SGL, not the fastest card around but the limiting factor will anyway be the network you are streaming over.

Remember that you shuld place the OS for such a system on a separate SSD and not the RAID data drive.


The server is housed in a Fractal Design Define R4, it has room for 8 3.5” drives AND 2 2.5” drives, standard motherboards and PSU. I also have it connected to a UPS so that sudden power loss or voltage spikes are less prone to damage the drives.


For multiple hdds its best to use RAID 5 or RAID 6, 5 only need one extra drive for backup, but you are only protected from a single drive failure. RAID 6 uses two drives for backup so if one drive fails your data are still quite safe while you are rebuilding the array (it can take a week to rebuild an array).


If you need 12 TB space you then need MINIUM 4 drives for RAID 5 (3 drives for data, 1 for backup in the RAID structure) but the best would be 5 drives in RAID 6 (3 drives for data, 2 in backup).


On the RAID card I have it was easy to expand the array (or change it from RAID 5 to RAID 6) online (meaning I could access the array and the files on it while it was migrating/expanding.

Hope that helps a little

 

[Jhelms]

Outstanding info guys! On my reading I was already leaning to raid 6 as well for my needs and data safety. I am leaning to just go 16TB for future expansion. So in R6, it seems I will need 6qty of 4TB drives as usable capacity is 2 less than the number of drives available. I found this page very helpful in quick learning. http://www.adaptec.com/en-us/_common/compatibility/_education/raid_level_compar_wp.htm I am surprised it takes so long to rebuild an array! Is this generally done automatically via software when a new / good drive is installed?

I have several AM1 5350 based systems that have been excellent for general use. The advice clearly points me to just build my own and I will base it off of the AM1 platform for low cost and high reliability / low power consumption. And agreed on the APS - already planned.

Good to know on the greens. I was looking more for a drive that does spin down though as the whole setup would not be used often. Just for streaming when home and data backup. If the reds are the ticket though as they are designed for NAS, then that is the direction I will go.

 

[Brusfantomet]

The raid card I have allows for a spare pool of drives that will be used to rebuild an array should a drive in the array fail automatically. If it rebuilds the array at ones if it detects the array have been damaged an a new drive is inserted I do not know

The Reds are greens with some FW trics (they awnser the controlles faster with “I am alive” messages) so that RAID controllers know the drives are good, I think it’s the same with the other “NAS” drives on the market, they are budget drives with some FW tweaks. I am sure @SuperSoph_WD (resident WD rep) can answer it more in depth.

I am personally using Segate NAS drives, mostly because they are 5900 rpm, so less noise then a 7200 disk. You can actually get them with some “Rescue” package now, where seagate will recover the data from a crashed drive for you if the disk dies in the first 3 years.

A AM1 5350 would be good I think, get a good MB with some solid caps then.

Yes for R6 and 16 TB you need 6 drives at 4 TB each.

For spinning the hdds down you just set the OS to pause the drives when not in use.

 

[ChristTheGreat]

1- No green, please use WD Red for TLER!
2- Remember that using Onboard RAID = using CPU usage, a little
3- No raid battery card with cache = Do not enable write-back caching, as if you crash while writing, you could get corrupted DATA.
4- if using Intel raid or Raid card with no battery, you can enable write-back cache when you need for big writing, instead, I think you can get between 40 and 70mb/s of writing, which is somehow enought (gbps will do max 125 over 1 single cable)
5- Raid 6 is not supported on Intel chipset, look for LSI megaraid card or other good card for RAID 6 support
6- Freenas likes to have alot of RAM using ZFS, so a minimum a 8gb of RAM for good performance (they say about 1gb of ram per 1TB, but anyway 8gb will be fine)
7- You could use a Windows 7 for file server, if you don't have 17 device accessing it. (Limited device/user share)
8- Raid 6 Is a very good option for redundancy with alot of drive.
9- If you need CPU performance over the name, don't take single core CPU. Look for Intel Penium dual-core that are cheap which would be enought fast!

As for backup, you will need to make a "bit torrent sync" or a manual/automatic script backup to make it easy as a backup, never worked on FreeNas, I don't know if there is a Eternal Site backup job of similar feature.

 

[VulkanBros]

+1 for FreeNas and WD Reds...we are using it for backups from our VMware farm....
10 x 4TB disks (in RAID-Z) and with ZFS filesystem in a rack with an core i5-4460 and 12 GB RAM.
The build in compression in ZFS is amesing .......

Write speeds over LAN is 90~110 MB/s (using HP ProCurve 2920 switches)

 

[Brusfantomet]

1- No green, please use WD Red for TLER!

Same for all other brands in my mind

3- No raid battery card with cache = Do not enable write-back caching, as if you crash while writing, you could get corrupted DATA.

A UPS shuld fix that (the computers writing or the lan streem will be broken before the UPS is empty, allowing the raid card to arrange the data on the drive and the whole computer to shut down correctly to avoid data loss

4- if using Intel raid or Raid card with no battery, you can enable write-back cache when you need for big writing, instead, I think you can get between 40 and 70mb/s of writing, which is somehow enought (gbps will do max 125 over 1 single cable)

Getting 115 MiB/s here using write back on a asus RT-AC66U

7- You could use a Windows 7 for file server, if you don't have 17 device accessing it. (Limited device/user share)

True, win server is nice an all, but no consumer/free antivrus or FW will not run on it, and the server stuff is a bit steep (cheapest is 150 usd a year) so using win 7 is a good option. If you are running win pro you can use remote desktop to manage the server

 

[Jhelms]

Great advice and info. In my mind, this setup will be directly linked to my HT for video playback. making it a monster HTPC / NAS for all users.

Will start researching raid cards here shorty.

 

[rtwjunkie]

@Brusfantomet Actually, AVG File Server Edition is quite affordable and runs on server 2008.

 

[ChristTheGreat]

Same for all other brands in my mind


A UPS shuld fix that (the computers writing or the lan streem will be broken before the UPS is empty, allowing the raid card to arrange the data on the drive and the whole computer to shut down correctly to avoid data loss


Getting 115 MiB/s here using write back on a asus RT-AC66U


True, win server is nice an all, but no consumer/free antivrus or FW will not run on it, and the server stuff is a bit steep (cheapest is 150 usd a year) so using win 7 is a good option. If you are running win pro you can use remote desktop to manage the server

Write-back cache is where you cache Data. 115 is good, but without you might get less, but might be enought for most people (writting at 60mb/s for me is enought over lan)

the UPS will protect only if there is an electric blackout. When the computer crash, the raid battery will remember what's in the buffer, so it can write to the disk.. Never will be able to do this on an onboard Intel..

Here'S an explanation:

It allows the raid card to remember what is in its buffers ( that hasnt been sync'd to disk )

Its very important for people who need high data integrity.. Or to save your DB from certain types of corruption..

(Basically whats on disk, is on disk - so thats safe.. The problem is when the OS thinks its on disk but its actually not and in a RAID card buffer)

When the server starts up again, obviously those buffers get flushed to the disks.. So you have a point in time correlation with your disks and OS..
( otherwise you will just loose information - like a few database records, which you will never know. )

A UPS help sure.. but its not safe enough.. ever decent RAID card should have a BBU (Battery Backed Unit)"

It just ensure that you have data integrity. Anyway, it depends of what usage it is, but for a 16TB, I would surely prefer having a BBU ona raid card..

 

[Brusfantomet]

@Brusfantomet Actually, AVG File Server Edition is quite affordable and runs on server 2008.

true, will probably try that, tank you.

Write-back cache is where you cache Data. 115 is good, but without you might get less, but might be enought for most people (writting at 60mb/s for me is enought over lan)

the UPS will protect only if there is an electric blackout. When the computer crash, the raid battery will remember what's in the buffer, so it can write to the disk.. Never will be able to do this on an onboard Intel..

True, but as the files on MY server is for conveinece only i do not see a OS crash as a big danger. for OP it might be different.

 

[ChristTheGreat]

Yup, that's why I just mention the point

I do not have a raid card at home, anyway, I have my OS backup on External drive and DATA replicated 2 days a week at my parent's home server (on the external drive)

 

[ne6togadno]

Great advice and info. In my mind, this setup will be directly linked to my HT for video playback. making it a monster HTPC / NAS for all users.

Will start researching raid cards here shorty.

if you are going to use freenas better not waste time on raid cards.

ZFS was designed to provide redundancy while addressing some of the inherent limitations of hardware RAID such as the write-hole and corrupt data written over time before the hardware controller provides an alert. ZFS provides three levels of redundancy, known as RAIDZ*, where the number after the RAIDZ indicates how many disks per vdev can be lost without losing data. ZFS also supports mirrors, with no restrictions on the number of disks in the mirror. ZFS was designed for commodity disks so no RAID controller is needed. While ZFS can also be used with a RAID controller, it is recommended that the controller be put into JBOD mode so that ZFS has full control of the disks. When determining the type of ZFS redundancy to use, consider whether your goal is to maximize disk space or performance: ....

http://doc.freenas.org/9.3/freenas_intro.html#zfs-primer

 

[T-Bob]

If it was me, I would use a hardware raid card and a raid5 or raid6 configuration with an extra drive as a hotspare.

 

[Brusfantomet]

if you are going to use freenas better not waste time on raid cards.

http://doc.freenas.org/9.3/freenas_intro.html#zfs-primer

But for the amount of disks needed (7 or more) you would need some extra S-ata controller cards on most normal motherboards

 

[Aquinus]

I would personally use MDADM or ZFS (with ECC memory) on something like this:
http://www.asus.com/Commercial_Servers_Workstations/P9AIC2550SAS4L/

...or for a bit more money, they have a C2750 variant with an 8c Atom.
http://www.asus.com/Commercial_Servers_Workstations/P9AIC2750SAS4L/

4 SAS breakout cables and 2 SATA ports means a max of 18 SATA drives in a mini-itx board. Sounds kind of like what you want.

Edit: On top of that you can add more of something else with that PCI-E 8x slot (@4x electrical).

 

[ne6togadno]

But for the amount of disks needed (7 or more) you would need some extra S-ata controller cards on most normal motherboards

am3+ boards are tipicaly with 6-8 sata3 ports and if you still need more you can do it with simple port expansion card. no need of raid card with fancy features just to run it in jbod mode.

 

[Jhelms]

Appreciate all the input guys. Lots to think about and more research to do!

 

[Jhelms]

Here is what I ended up with and why:
- I am virtually out of space with my current setup (less than 50gb left) and need to get started on this now. I decided to go at this slow but with a plan / path. Decided not to go to crazy right now as I am about to buy a house and keeping funds tight...
- Ordered an 8tb my book duo that will be configured in raid 1 to be connected via USB 3.0 to my PC which will give 4TBish of space that I need right now. This unit contains two RED 4TB drives. This is a decent deal as I get the 4tb reds and the raid station which makes the raid station only $50 in this combo. (379 for the two reds and raid station / individual 4tb reds are $166ea)
- I also ordered in a 4TB red as an emergency backup that will be powered off most of the time in a docking station. So overall I have 3 qty of 4TB reds currently on the way
- This will hold me over until mid summer in which I will buy 3 additional 4TB red drives to create the raid 6 array I am after. This will then give me the 16TB of space I am looking for overall

Just wanted to keep you guys updated on my path as I appreciate all the help!

 

[TRWOV]

I have the following setup on the office:

1. FreeNAS server with Athlon 5350, 4port RAID card in no-RAID mode, 16GB of RAM and 4 RE3 1TB drives in RAIDZ2 (basically RAID5 with write hole protection). Total capacity 2TB. 450w UPS.

2. FreeNAS server with C60, 8GB of RAM, 3 RE3 1TB drives in RAIDZ1. Total capacity 2TB. 250w UPS.

3. 2TB drive on a external enclosure.

I have setup the 5350 server to replicate on the C60 server every 2 hours so that if the 5350 goes down for some reason I can just switch to the C60 at any moment and I make weekly backups on the 2TB external which then are saved back at home.