Viability of RAID, long term

[TheLostSwede]

For RSTe, a couple hours-ish with my 1TB WD Blacks. Bigger drives are likely going to take longer. It depends on how much is going on while the rebuild is occurring, but I usually can use the machine while it's occurring. Any modern RAID implementation is pretty fast when it comes to RAID-5 though. It's not like a current gen CPU (or even my SB-e for that matter,) has to work very hard to XOR the parity bits to figure out what the missing blocks are supposed to be.

Tried to rebuild a RAID for test purposes when I worked at QNAP. That was with some kind of Atom CPU and took forever, as in a couple of days..

 

[Jetster]

I'm really happy with TrueNas, formally FreeNas. You don't have to worry about a controller failing. ZFS and a RAID 6 so two drives would have to fail.
If a controller fails or a board you just replace it and you are bock up running.
Yes it is backed up somewhere else. 8 Tb of data

 

[Aquinus]

Tried to rebuild a RAID for test purposes when I worked at QNAP. That was with some kind of Atom CPU and took forever, as in a couple of days..

Yeah, it really depends on the implementation. RSTe has actually been one of the fastest implementations I've used. AMD's RAID is absolutely terrible (over 8 hours on at least on the 700 and 800 series chipsets, I actually eventually gave up and just used mdadm) and nVidia's when they used to produce nForce chipsets were okay (4-ish hours.) mdadm is almost on par with RSTe as well. Rebuild performance is decent on decent hardware and read performance on RAID 5/6 is exceptional. Write performance was a little lacking, but I think that has more to do with byte alignment of the blocks on the disk with the stripes of the RAID though.

All in all, I can't complain about RSTe or mdadm.

 

[Operandi]

I'm really happy with TrueNas, formally FreeNas. You don't have to worry about a controller failing. ZFS and a RAID 6 so two drives would have to fail.
If a controller fails or a board you just replace it and you are bock up running.
Yes it is backed up somewhere else. 8 Tb of data

TrueNAS is pretty great but I'll point out that it is pretty picky about the hardware it likes to run on. You can get it to work with a lot of controllers but its designed to be used with card/controller that works in pure HBA mode otherwise you run the risk of logical corruption. Also by its nature every with ZFS every drive in the array needs to be identical which isn't really a downside of ZFS but there are other options out there that are more flexible. Basically TrueNAS is designed to run on enterprise server hardware.

Another option to look at would be unRAID. It works completely differently than TrueNAS and ZFS and allows you build the array with a mix and match of drives and build up the array over time. Its also far more flexible on the hardware it will happily run on (pretty much anything).

You can also look at Xpenology which is basically a project that lets you run Synology on your own hardware. I have no experience with it but people seem to like it.

 

[MentalAcetylide]

There's just too much shit that can go wrong with RAIDs, much of which can be mitigated in a controlled commercial IT setting when compared to consumer desktops. It doesn't happen very often, but I do know that my system crashes or needs a hard/forced reboot a lot more frequently(due to software) than the one I use at work. I don't know about software RAID, but forced reboots & system crashes can be a big problem for hardware RAID. Quite frankly, if you don't need RAID, save yourself a ton of potential headaches and just do regular backup copies instead. While they can be beneficial depending on your needs, when things go wrong, it can be a pain in the ass; especially if you're not familiar with them.

Anyway, as long as you back everything up on a separate drive, you can run whatever RAID floats your boat without any worries of losing everything due to a system crash or hardware failure.

 

[newtekie1]

There's just too much shit that can go wrong with RAIDs, much of which can be mitigated in a controlled commercial IT setting when compared to consumer desktops. It doesn't happen very often, but I do know that my system crashes or needs a hard/forced reboot a lot more frequently(due to software) than the one I use at work. I don't know about software RAID, but forced reboots & system crashes can be a big problem for hardware RAID. Quite frankly, if you don't need RAID, save yourself a ton of potential headaches and just do regular backup copies instead. While they can be beneficial depending on your needs, when things go wrong, it can be a pain in the ass; especially if you're not familiar with them.

This just doesn't make any sense. RAID, with the exception of RAID 0, makes data storage safer. Home use or not, data storage on a RAID 1/5/6 is safer than data storage on individual drives.

Hard reboots aren't a problem for RAID. You aren't going to lose all your data because you have to reboot the system, it doesn't work that way. Worst case is if you have write-back cache enabled without a BBU. Then the data in the write cache will be lost. But that is just mitigated by using write-though cache instead, which is slower for writing but doesn't have the issues of data loss in the case of a unexpected shutdown.

 

[Operandi]

This just doesn't make any sense. RAID, with the exception of RAID 0, makes data storage safer. Home use or not, data storage on a RAID 1/5/6 is safer than data storage on individual drives.

Hard reboots aren't a problem for RAID. You aren't going to lose all your data because you have to reboot the system, it doesn't work that way. Worst case is if you have write-back cache enabled without a BBU. Then the data in the write cache will be lost. But that is just mitigated by using write-though cache instead, which is slower for writing but doesn't have the issues of data loss in the case of a unexpected shutdown.

I think the point was any kind of RAID configuration is complex. If all you need is another copy of the data a 10TB (or whatever capacity you need) USB hard drive is way less complex and less likely to fail than an array of 4+ drives not to mention the hardware managing it. A NAS like Qnap or Synology or building your own TrueNAS or unRAID is resilient to downtime and can do all kinds of cool shit but there is no arguing its complex and has multiple points of failure.

 

[dragontamer5788]

Back when computers were slower, all of those calculations were too expensive to be conducted by the CPU. So you wanted a specialized ASIC to run the RAID5 checksum algorithms (aka: a RAID controller). But today, modern CPUs have so much spare compute power, you really don't want to be using a RAID controller anymore. In fact, many RAID systems on cheaper motherboards are just software RAID (a device driver that performs this RAID calculation, except it only works on a given motherboard that the device driver was made for). As such: RAID is just a bad idea: if the motherboard dies, there's no guarantee that a future motherboard knows how to read your RAID collection.

In contrast, a modern filesystem like ZFS (Linux/BSD) or ReFS (Windows enterprise storage spaces: "REsillient File System"), will be able to be rebuilt on a new motherboard automatically. If your "RAID" is all software anyway, might as well embrace the software-based methodology to the logical extreme and just use these features afforded by the operating system.

--------------

So when people today talk about "RAID5", they really mean running ZFS in 1-disk parity mode or Storage Spaces in 1-disk parity mode. I don't think people really mean buying a hardware RAID system anymore (ex: LSI MegaRAID or whatever). These software only solutions have been getting better and better (moreso on Linux, but ReFS / Windows Storage Spaces still has nice features worth talking about).

I recommend building a system with ZFS: be it Linux ZFS or BSD ZFS (Xigmanas is what I use) and really playing around with it. I also recommend 4-hard drives in striped+mirrored (aka: RAID10-like setup), which will only fail if 2x hard drives fail. striped+mirrored works efficiently in all software systems (Windows Storage Spaces and Linux), so its a good default. Linux's RAID5-like and RAID6-like software setups are known to be pretty efficient (Windows Storage Spaces not so much so). If you're going beyond 4-hard drives, working with Linux for 2x parity drives on say a 8-HDD setup is likely going to be less of a headache than 2x parity drives on Windows Storage Spaces. Windows works, but something about its algorithm is a good bit slower IIRC in this case.

---------

The main advantage of ZFS or ReFS is that Linux/Windows/BSD is more aware of the disks. In particular, the "ZFS Scrub" command will automatically scan all data and check for bitrot (and automatically correct any issues that come up). If you were using a hardware RAID system like LSI MegaRAID, Linux wouldn't necessarily know how to scrub / rebuild (it'd be part of the controller). There are also advanced software features like ZFS snapshots available.

 

[Operandi]

Yeah hardware RAID isn't really a thing anymore. I'm sure Qnap or Synology are doing it in their software.

If the OP (or anyone else) builds a NAS there are tons of LSI controller cards out there and you certainly can use them to build an array what you really want is one that is configured to work like pure HBA that just presents the disks to the system and let software (TrueNAS, unRAID) handle all the parity and rebuilding stuff.

 

[dragontamer5788]

Yeah hardware RAID isn't really a thing anymore. I'm sure Qnap or Synology are doing it in their software.

If the OP (or anyone else) builds a NAS there are tons of LSI controller cards out there and you certainly can use them to build an array what you really want is one that is configured to work like pure HBA that just presents the disks to the system and let software (TrueNAS, unRAID) handle all the parity and rebuilding stuff.

And dedup, and NVMe cache, and scrubbing, and snapshots, and...

Seriously. Software solutions these days have so many features that hardware RAID is no longer a contender IMO. Especially in this day and age where we can just buy an 4-core/8-thread chip under $200 (Ex: AMD Ryzen 3 3300X). Sure its Zen2 instead of Zen3, but you probably don't need a Zen3 to run just a NAS workload.

EDIT: So to be clear: I'm recommending a 2nd computer to serve as a NAS. If you want the NAS-storage to be used on your main computer, carve up a bit of the NAS volume into an iSCSI and have your main desktop connect to the NAS remotely. If you don't like the idea of running your hard-drives over a network (despite 1Gbps ethernet supporting ~100MB/s transfer speeds), try Windows Storage Spaces (which would keep the entire solution local to your Desktop).

 

[ShiBDiB]

It depends what you're storing (which I didn't see stated anywhere). Unless it's something super sensitive, just use a legitimate cloud provider as your backup.

As can be seen by this cluster of a thread, RAID is not the answer for a no bs backup solution.

 

[Hypnotized621]

Idk why people keep going on about RAID0 and losing everything because of a HDD or SSD failure. I mean, if you only had one HDD or SSD and that fails the same thing happens.

 

[Ahhzz]

Idk why people keep going on about RAID0 and losing everything because of a HDD or SSD failure. I mean, if you only had one HDD or SSD and that fails the same thing happens.

No one in here is going on about a Raid0 and losing everything. Everyone here understands what Raid0 is, and its limitations, and purpose.

 

[ThrashZone]

Hi,
I personally keep things simple as possible and backup wise works for me
Only raid I know of comes in a can to kill bugs

 

[newtekie1]

I think the point was any kind of RAID configuration is complex. If all you need is another copy of the data a 10TB (or whatever capacity you need) USB hard drive is way less complex and less likely to fail than an array of 4+ drives not to mention the hardware managing it. A NAS like Qnap or Synology or building your own TrueNAS or unRAID is resilient to downtime and can do all kinds of cool shit but there is no arguing its complex and has multiple points of failure.

This is exactly why were are saying RAID isn't a backup. If you just need a backup of your data, yes get a 2nd drive and back your data up to that drive. RAID serves a different purpose.

It depends what you're storing (which I didn't see stated anywhere). Unless it's something super sensitive, just use a legitimate cloud provider as your backup.

As can be seen by this cluster of a thread, RAID is not the answer for a no bs backup solution.

Cloud solutions aren't always feasible. I mean, I have 20TB of data that I want to have instant access to, I also have a 1TB per month data cap. Both situations prohibit me from just storing everything in cloud.

 

[Athlonite]

Cloud solutions aren't always feasible. I mean, I have 20TB of data that I want to have instant access to, I also have a 1TB per month data cap. Both situations prohibit me from just storing everything in cloud.

That's a pretty big porn collection lol

 

[OneMoar]

That's a pretty big porn collection lol

Its not all porn some of it is furry hentai

 

[newtekie1]

That's a pretty big porn collection lol

Don't be jealous, only half of it is porn. The rest is the non-nude pictures of your mom.