RAID6: Samsung or WD?

[FordGT90Concept]

There's a specific issue with Western Digital non-RE series drives and support for TLER. When the time to recover from an error is too long then the drives will act as if they're unresponsive and be dropped from the RAID.

In the past, TLER could be enabled on all of Western Digital's drives. But to move RAID users to the RE series they imposed a block on enabling TLER on desktop drives.

So, yes, there are limitations to which drives can be used with which controllers. Some controllers can get around this problem. I am unfamiliar with the controller the OP mentioned.


Western Digital
Wikipedia

Sounds to me like people shouldn't be buying Western Digital drives then. Crippling drives just so you have to buy a more expensive model is business conduct that should be shunned.

 

[theeldest]

What was happening before the block was IT departments buying dozens to hundreds of Black drives, adjusting the TLER value and using them on controllers with stricter timeouts. To WD, each box (20 drives) of Blacks sold where RE should have been used was a loss of ~$2K. With that kind of savings, it was easy to have spares on hand to cope with the marginally higher failure rate. WD didn't block TLER to squeeze consumers for more coin, they did it so IT would fall in line with their product tiers

Sounds to me like people shouldn't be buying Western Digital drives then. Crippling drives just so you have to buy a more expensive model is business conduct that should be shunned.

I'm actually OK with companies artificially differentiating their products. Take i7 vs i5 & hyperthreading. K series vs Locked.

It generally helps to drive prices apart. If you don't need the features, you pay less. If you do, you pay more.


The reason I switched from WD to Samsung in my fileserver is the lack of 'enterprise' quality 5400rpm drives from Western Digital. Average filesize on my server is 4GBs. I don't think I really need blistering access times if I can save the power.

Heck, only Seagate goes down to 7.2k rpm in the 15mm form factor. Everyone else is 10k or 15k.

Where are the power efficient 'enterprise' drives?


But back to the OP:
Do your enclosures need actual laptop sized drives? is performance important?

I think the two best options are the seagate constellation if performance is important and your enclosure can handle the 15mm drives.

Otherwise the best option is the cheapest non-WD drive.

 

[FordGT90Concept]

Enterprise have more stringent quality controls that result in a longer average lifespan (substantially longer usually). That is differation enough for everyone except Western Digital, apparently.

Your example, for instance, comparing i7 and i5. Intel's quality control process results in binning: the chips that operate the most flawlessly end up as i7 while those that operate at less than optimal end up as i5.

Western Digital is taking a perfectly fine product and intentionally breaking a function of it so it won't work with industry standards. That's damn near criminal.

Where are the power efficient 'enterprise' drives?

Hybrid or solid state. Enterprise drives are designed for both performance and longevity. 5400 RPM doesn't fit in that first category.

 

[Aquinus]

Western Digital is taking a perfectly fine product and intentionally breaking a function of it so it won't work with industry standards. That's damn near criminal.

Stop making those claims unless you've actually used a Western Digital RE4, an enterprise grade disk designed for RAID or the Caviar Blue which has TLER. I think you should learn a little bit about WD's offers before you start making such claims. If you take a look at their consumer grade hard drives you will see that it basically breaks down to this (this is off their website).

WD Caviar Blue
Performance and reliability for everyday computing.

WD Caviar Green
High-capacity, cool, quiet, eco-friendly storage.

WD Caviar Black
Maximum performance for power computing.

Okay, that's pretty simple and straight forward, right?

Now SATA enterprise offering:

WD RE4
Massive capacity, 64 MB cache, 1.2 million hours MTBF, and a 5-year limited warranty, WD RE4 drives offer an ideal combination of high capacity, optimum performance, and 24x7 reliability for enterprise applications.

I think that is the key phrase you're looking for, and honestly, I turned off Patrol Read on my RAID-5 and it's running like a dream, with 1 Hitachi drive, 1 WD Black from 2012, and 1 WD Green from 2008. I don't think you have to worry about it too much and honestly, I've had nothing but good luck with WD.

Western Digital got an RMA drive for work yesterday afternoon (it was a WD green from 2009), they had the replacement out and shipped the next morning. If that isn't service, I don't know what is.

Thanks for dealing with my rant, I just had to defend WD as their a great company with great customer service and plenty reliable drives. We have 12 caviar blacks at work from 2007, they're all still running and haven't skipped a beat. They've been running 24/7 since we got the servers.

Where are the power efficient 'enterprise' drives?

I'm not convinced that "Enterprise" and "power efficient" belong in the same phrase. In all honesty, when I want to copy a database, I don't care if it's eating 5 watts or 15 watts, I just want the database to copy as fast as it can as reliably as it can.

 

[Albuquerque]

Nice, I love the conversation that has ensued since last I checked

The array in question is pure data storage; a few video streams might come out of it simultaneously but that's going to be rare. A single one of these drives could maintain two 1080p MKV datastreams without issue, eight of these spindles in a RAID6 config is going to far exceed my usage pattern (in terms of overall performance).

That data is not life-critical, but it is data that I really do not want to lose. We're talking pictures and video of my child's first moments in this life, electronic copies of financial or legal documents (we have paper copies too, also in a safe), and other such detail. Thus, RAID6 is where I would like to be. We also have some cloud storage that will maintain the most 'critical' of data, but I really don't need my array taking a huge dump and taking all my 'slightly less than critical' data with it.

So, it goes like this: I need a moderate to high amount of data integrity, I need low to very low power draw at idle (the overwhelming majority of this box's life will be at idle), I want a reasonable amount of storage space, I want it to take up a tiny amount of physical space (HTPC size) and I do not require a significant amount of speed. RAID6 on laptop drives seems to fit most of that, so long as I don't get owned in the face by RAID compatibility issues (TLER being obvious.)

I think that should clear up most of the questions

 

[FordGT90Concept]

Stop making those claims unless you've actually used a Western Digital RE4

My server has an RE3 replacing one of the Barracuda ES drives that failed.

My computer has two Barracuda, non-ES, drives, that have been RAID0 for over 7 years. My server has two non-ES drives that have been RAID1 for 5 years.

I buy nothing less than Caviar Black from WD and I have one in my server external backup.


@Albuquerque: Hot-spares are very important. If you are doing 8 in your RAID6, that doesn't leave any room for a hot-spare. The controller supports it.

 

[Albuquerque]

RAID6 requires two drives to fail before I lose data; I'm not sure that I would want an additional hot spare on top of that. I already have another 1TB drive at home that I could immediately swap in on a single drive failure; if the stress involved in the rebuild takes down the entire array, well then I'm screwed aren't I? A hot spare wouldn't save me from that anyway though.

 

[FordGT90Concept]

I have a similar Highpoint RAID card. It keeps track of the SMART data and if a drive is marked by SMART as a failure, it automatically replaces the drive with the hot spare. Once the hot spare has been initialized, it sounds an audio alarm. Look at the errors in the management software and it will tell you the model/serial number for the failed drive. Yank it out and in it's place, put in another hot spare.

Hot spares basically mean the window in which the RAID is compromised is isolated to the time it takes to initalize it (less than an hour likely).

You're right that you don't need it. In fact, I would probably put those resources (spent on spares) into an external backup should the volume catastrophically fail.

 

[Albuquerque]

I could probably just build a messaging system in Windows task scheduler to generate an email event when the system event log data is captured from the Highpoint controller -- if it exposes such an event in the system event log.

I know the controller itself also has the ability to do messaging; I had already considered looking into that anyway.

 

[theeldest]

To me: RAID 6 = RAID 5 with an active hot spare.

Given your use, the Samsung you mentioned in the original post is probably the best bet. (lower power draw than the Western Digital AND you don't need to worry about the WD TLER issue; win, win)

I've got 5400 rpm Samsung drives (though they're 3.5") in my server (backups and XBMC) and they give no problem for video streams. Given the larger number of data drives (your 6 vs my 3) you'll certainly have no problem.


And @ aquinas:
Not all enterprise activities really need performance. Off hand, I can think of email archiving and video surveillance systems. For both of these you want the most storage/year/dollar. ie, you'd pay a bit more for higher efficiency drives.

I think we'll go that direction as CIOs/IT Directors/etc are becoming more conscious of the cost of running things in addition to cost of buying equipment.

Heck, the big server OEMs are putting 80 Plus Titanium PSUs in systems this year. Large organizations stand to save quite a bit of money by cutting OpEx.

Until solid state tech becomes competitively priced per gigabyte we'll continue to see tiered storage. (SSD -> 15k/10k SAS -> 7.2k -> 5.4k?). It's not too far out of the question to think manufacturers would add that last step.

 

[Albuquerque]

To that tiered data point -- my RAID1 array for the OS and apps will be using the Z77 chipset, of which I'll be leveraging the SSD cache option. A lot of the build-your-own-hybrid approaches are making a LOT of sense now, both for home use and enterprise.

 

[theeldest]

To that tiered data point -- my RAID1 array for the OS and apps will be using the Z77 chipset, of which I'll be leveraging the SSD cache option. A lot of the build-your-own-hybrid approaches are making a LOT of sense now, both for home use and enterprise.

The Intel SSD caching does actually work really well. I'm using it to accelerate a RAID 10 in my desktop. I can't imagine that z68 vs z77 really makes much difference.

The only thing to consider is Maximized vs Enhanced (the two caching options). The difference is whether writes are cached (maximized DOES cache writes).

For what you're doing, it's a toss up whether one over the other would be better. Your cache will have lower throughput than the array for sequential writes and would thus slow it down. But the caching algorithm already takes into account large files and will not cache them. (so if you're copying a 1Gig + file, it'll skip the cache, even if you're using it in Maximized mode).

Just FYI.

 

[Albuquerque]

I haven't decided which caching mode I'm going to use yet. Maximized is certainly an option as the box plugs into a UPS and I have the software configured to properly acknowledge power issues and shut the box off gracefully during extended blackouts.

As for performance: I have the OS + Apps array (dual 320GB WD Scorpio Blacks in RAID1) and the Hyper-V array (dual 750GB WD Scorpio Blacks in RAID1) already running right now in the new box. So far, the OS performance is fantastic even without the caching. The whole system goes from power-button-click to CTRL+ALT+DEL screen in about 15 seconds. Given what I'm seeing right now, it may be more advantageous to enable the caching on the Hyper-V array to get all the VM's booted quickly.

The obvious solution is to tinker with it

 

[theeldest]

I haven't decided which caching mode I'm going to use yet. Maximized is certainly an option as the box plugs into a UPS and I have the software configured to properly acknowledge power issues and shut the box off gracefully during extended blackouts.

As for performance: I have the OS + Apps array (dual 320GB WD Scorpio Blacks in RAID1) and the Hyper-V array (dual 750GB WD Scorpio Blacks in RAID1) already running right now in the new box. So far, the OS performance is fantastic even without the caching. The whole system goes from power-button-click to CTRL+ALT+DEL screen in about 15 seconds. Given what I'm seeing right now, it may be more advantageous to enable the caching on the Hyper-V array to get all the VM's booted quickly.

The obvious solution is to tinker with it

In any event, sounds like you have some fun times ahead.

Be sure to give us an update when you get everything settled on!

 

[Albuquerque]

Will do. No matter which drives I use, I'll pummel them with way too many writes over the course of several days to ensure one (or more) don't go astray due to TLER or other issues. Once I feel like I've proven them sufficiently stable, I'll start building the guest OS instances and can really get into the performance tuning.

One thing I did right: the whole box at idle (the four drives and two 120mm fans spinning but the KVM disconnected) consumes 32W at the wall according to my Kill-a-Watt E3. It's an MSI Z77 board, an i5-3570k, 16Gb of ram, an LG Bluray drive and the highpoint RAID card with no drives attached and all connected to a PC Power and Cooling 500W 80+Bronze modular unit. I even have Aero glass and audio enabled on my Win2K8 R2 OS, which I'm sure adds to the tally just a tad.

Not bad

 

[theeldest]

Will do. No matter which drives I use, I'll pummel them with way too many writes over the course of several days to ensure one (or more) don't go astray due to TLER or other issues. Once I feel like I've proven them sufficiently stable, I'll start building the guest OS instances and can really get into the performance tuning.

One thing I did right: the whole box at idle (the four drives and two 120mm fans spinning but the KVM disconnected) consumes 32W at the wall according to my Kill-a-Watt E3. It's an MSI Z77 board, an i5-3570k, 16Gb of ram, an LG Bluray drive and the highpoint RAID card with no drives attached and all connected to a PC Power and Cooling 500W 80+Bronze modular unit. I even have Aero glass and audio enabled on my Win2K8 R2 OS, which I'm sure adds to the tally just a tad.

Not bad

That's pretty awesomely low power draw. What case are you using for this?

I'm considering upgrading my XBMC/fileserver to get lower power draw and you've given me some good ideas. I'd also like a smaller case (preferably an HTPC case but they're too wide for where I'd like to put it--I have 13"x13")

 

[Albuquerque]

I'm using this case: http://www.silverstonetek.com/product.php?pid=314

I ordered the eight 1TB WD drives on Thursday via Amazon Prime, turns out that 2-day free shipping also includes Saturday! I have all eight wired and connected, started the RAID initialization on Sunday and it finished late last night. I loaded up WHSv2 on an Hyper-V instance with the OS as a VHD file on the OS+App array for now (still missing one 750GB drive for the virtual machines, should arrive tomorrow) and the RAID6 data array direct-linked through the VM.

I loaded up a 100GB test file on the single 750GB drive and have a robocopy process that is continually re-writing that file all over the RAID6 array since about 6:30am PST this morning. It's noon now, and it has yet to toss a drive after six hours of continuous writes... So far, TLER on laptop drives appears to be a no-show, or at least it's not impacting the Highpoint 2720's ability to do RAID6 correctly.

I can't see exactly how many instances have run (I didn't think to write that into a log) so I cannot say how fast the array is. I can tell from Resource Monitor that the speed limit is the read rate of the 750GB drive, the RAID6 array is apparently capable of more than ~110GB/sec writes sustained. I'll do some ATTO or something similar when I get home and post some pics.

It appears that, with the Highpoint card loaded up and eight drives stacked in, the box is going to idle right around the 40W mark. Loading up WHSv2 (from an ISO file) onto the HyperV instance "peaked" the box out at around 46W. Before I left the house, the E3 device was recording around 43W running the continuous copy from 750GB drive to the RAID array.

I should've bought a smaller PSU

 

[theeldest]

I'm using this case: http://www.silverstonetek.com/product.php?pid=314

That's a FINE looking case. I might have an upgrade in my future ...

So far, TLER on laptop drives appears to be a no-show, or at least it's not impacting the Highpoint 2720's ability to do RAID6 correctly.

Good news gets better!

I'll do some ATTO or something similar when I get home and post some pics.

For disk benching head over to this thread: http://www.techpowerup.com/forums/showthread.php?t=166005

I should've bought a smaller PSU

What size did you use?

 

[Albuquerque]

That's a FINE looking case. I might have an upgrade in my future ...

I put 3.5" -> dual 2.5" converter in every available 3.5" bay for a total of ten 2.5" drives, and then a 5.25" -> 3.5" -> dual 2.5" converter below the BDRW drive for another pair. This gives me a total of thirteen 2.5" slots, which is an immense amount of storage space for such a small chassis. And it's DEAD quiet, even with the included fans at 100% speed.

What size did you use?

PC Power and Cooling 500W 80+ Bronze modular unit: http://www.amazon.com/dp/B0064XAJ30/?tag=tec06d-20

It got excellent reviews on Johnny Guru for efficiency and power regulation. And the rebate made it cheap Modular cables also keep the case clean; I'll take some pics of the inside.

 

[Albuquerque]

eldest, I couldn't get your ioMeter to stop complaining about sector size. I actually can't spend time on it tonight, as I had a cranky 7 month old and a tired wife to take care of. I did snap some pics and ATTO benches though:

The E3 meter showing 35W at idle:

The innards:

ATTO run from inside the WHS virtual machine (so the array volume is "passed through" Hyper-V, which gives interesting and slightly odd results)

ATTO run from the host operating system, so it produces a far more reliable / stable result

And snapshots of the configuration pages of the Highpoint 2720SGL array controller

Stats on the rig:

• MSI Z77-GD45 board
• i5-3570k processor (default settings)
• Silverstone PS07B case
• PC Power and Cooling 500W 80+ Bronze MKIII modular power supply
• LG 12x BDRW/DVDRW
• 2 x 8GB Mushkin value ram DDR3-1333 (undervolted to 1.35v)
• 2 x WD Scorpio Black 7200RPM 320GB drives on the ICH10-R RAID1 for OS+Apps
• 1 x OCZ Agility 3 240GB SSD for acceleration of the OS+Apps volume and scratch space
• 8 x WD Scorpio Blue 5400RPM 1TB drives on the Highpoint 2720SGL RAID6 for data
• 1 x WD Scorpio Black 7200RPM 750GB drive on the ICH10-R for VHD storage (second drive hasn't arrived yet to build the RAID1 array)

I think that about covers it.

 

[Albuquerque]

Aww, nobody wanted a low-power home server? I thought it came out well, especially worth noting that the WD drives haven't committed suicide on the RAID6 array too.

 

[Albuquerque]

It's been almost six months since I posted my last update in this thread; I figured someone might stumble onto this thread while researching and what to know how it really turned out.

A few months ago, Highpoint released an updated web interface utility that can now show the SMART status of all the drives connected to the controller. It just so happens that two of the drives reported SMART correctable read faults over the months. Despite those errors, the array has never once gone into a "degraded" or "failed" state. This either points to a favorable TLER capability of the WD Scorpio Blue laptop drives, or else a very "forgiving" capability of the Highpoint RAID card.

Both errors were logged, but both were corrected without any array state change. In fact, that array currently has an uptime of over 130 days, in which time it has faithfully served as the data volume for my WHSv2 virtual machine. That means it has downloaded and played back countless gigabytes of video and audio files in wildly varying formats and bitrates, along with 130 nights worth of backups from the household computers with WHSv2 clients installed. it also serves as the file target for backups of the host server.

So, after six months of full operation, I am comfortable in claiming that the Highpoint 2720SGL SAS RAID controller is 99.999% compatible with the WD Scorpio Blue 1TB laptop drives in pretty much any RAID configuration that would make sense to use. No TLER fallout, no missing drives in the array, and smooth sailing even when SMART reports issues.

Anyone considering a low power, low profile, inexpensive and large data volume set should at least consider this as an option IMO.

 

[snabe]

thanks

just registered to the forums to thank you for sharing your experience
found the thread looking for the same idea: safe storage+low power+low cost
the rocketraid 2720sgl has still no rivals in terms of price/performance
i decided to go with 8x hitachi travelstars because they have an 3y warranty
and are a bit cheaper too, sata3 as bonus
the disks are installed in a supermicro CSE-M28SAB enclosure:

http://www.supermicro.com/a_images/products/Accessories/CSE-M28SAB-OEM.jpg

unbeatable small form factor, but pricey and not really compatible to to highpoint's enclosure management (overheat/fan fail led does not work despite supermicro sas cables with sideband used)

it was a pain to get the controller to work on my msi k9nd board but the advantage of
a fake raid controller is: you dont need controller's bios to configure it (option rom issue)
flashed the firmware via pcie hot plug (booted to dos without controller, plugged it in live and flashed firmware 1.5)
works like a charm
configuration via web tool: 8x 1tb in raid6, 4k sectors and 64kb stripes, write back and disk cache on
windows auto aligned the volume at 128mb or something, seems ok

performance was really good with an empty array: nearly 600mb/s read and 500mb/s write
not bad for a 'hardware assisted' raid and cheap laptop drives

but here are some real world scores with array 50% filled:

http://img835.imageshack.us/img835/4651/attow2k8r2arrayfilled.jpg

also noticed write performance decreased a bit after switching from w2k3x64 to w2k8r2, despite win7 / server 2008 driver is newer
and the array gets slower with more data on it - like expected

seems the (cpu dependend) performance is suffering from processor's power saving features
write speeds go down if i use the balanced power profile in windows

when i bought the drives (4 each from different distros) i expected at least one drive with quality issues...
but no all 8 disks run without any smart event since 3 months
maybe the hitachis were a good choice


i'm sorry for necroing this thread but i hope some more people with low power/low cost arrays will report here...
i'm also curious about your experience with the wd disks

greets from germany

 

[Albuquerque]

Sooo, it's been a year. Guess what? One of my server drives died.

But guess what else: it wasn't one of the RAID6 drives I have two other RAID1 volumes on that server, one for the OS + apps and another for all the virtual machine files. One of the drives for the virtual machine files is what died, the horrible click of death is what ultimately killed it.

Funnily enough, it was a WD Black drive too. Meh. The cheaper, certainly-not-RAIDable WD Blue drives are still running strong as ever, quietly and efficiently taking care of business.

I'm replacing the dead drive with a new pair of HGST 1TB 7200rpm drives instead, just because I need the extra room.