RAID spontaneously degrading (Intel RST)

[Violet_Shift]

Hello everyone.

I'm having a few problems with my RAID, and given that this is the second time this has happened I feel I'd rather ask for help rather than blindly go replacing disks.

Last night while I was playing a game (loading from the RAID), the game froze and crashed, and RST gave me an alert saying that an error occurred. When I restarted the PC, RST tells me it's rebuilding.

Once the rebuild was done all seemed fine until the next day when I started the computer it reported the RAID as in "Rebuild" state and I go through that again... only to have RST again decide to rebuild the blasted thing less than an hour ago.

This first happened a few months ago, and in the end I replaced the disk in question to spare myself the aggravation. Seems it was only a stop-gap solution!

The thing is it doesn't seem there's anything wrong with the disks that "failed". They work perfectly fine up until RST decides it's rebuild time again.

I'm using a RAID-5 with four 3TB disks. Terrified to even use the computer now, given that I'm one spontaneous "disconnect" away from losing 4+ terabytes of data. (and given that I'm in the mood for gaming for the first time in a while!)

I really am a noob at this so if anyone wants to help me and anything complicated is involved can they please take it slowly with me

 

[Assimilator]

First principles: do you have the latest version of Intel RST installed? The latest as of this writing is 14.6.0.1029 - download and run the SetupRST.exe executable from here.

Secondly, are all the disks you're using from the same manufacturer and model? Mismatched disks can give very subtle and difficult-to-solve problems.

Next, are the disks you're using certified by the manufacturer for use in RAID volumes? Some disks, particularly the "Green" models, aren't suitable for use in RAID because they behave in a manner that the RAID controller doesn't expect. For example, if a Green disk goes into power-saving mode, and the RAID controller tries to read from it, the Green disk may fail to wake up fast enough to read that data, and the controller will assume this is because the disk is going bad. If this happens enough times, the controller will drop the disk from the array.

Double-check the SATA power and data cables. If possible, try swapping them for cables you are certain are good. I have personally experienced an issue with a bad data cable that didn't affect the data on the drive, but did make the RAID controller very upset. If it's always the same drive that seem to go "bad", that's a strong indicator that that drive's cables are actually the problem.

If none of the above are helpful/relevant, the disks may not be working as well as you expect. Install a SMART monitoring tool like Hard Disk Sentinel to check and verify your disks' health.

Should SMART show that your disks are all good, it doesn't mean you're out of the woods. Hard disks have a nasty habit of claiming they're working perfectly, up until they day they fail. At this point I would recommend getting hold of your disk manufacturer's low-level diagnostic utility (e.g. Seagate's SeaTools) and running it against your drives. If those utilities find a drive defect, they will generally guide you through the process to RMA the drive.

 

[FordGT90Concept]

RST should provide the serial number and model of the hard drive that failed/caused degradation. If it doesn't/isn't, I'd consider replacing the SATA cables and see if it still does it.

 

[brandonwh64]

Same thing happened to my new SSD raid 0 setup. It worked fine for about a week then I booted up and the bios raid screen shows "error occurred" but it still booted into windows then RST showed that one drive had failed. Its kinda odd that if a drive was failed that it would not boot into windows since its a raid 0 stripe but I could be wrong.

 

[Athlon2K15]

Id suggest if you are going to dabble with RAID use hardware RAID and not the intel onboard shit. Only thing you can really do is get your data off those drives before RST throws a fit, possibly changing SATA cables like FORD said above.

 

[brandonwh64]

I am actually pulling my second SSD and I installed a 4TB hard drive for storage/game install.

 

[Disparia]

Which drives Violet_Shift? There may be updated firmware available for them.

Awhile back I had some Seagates that didn't get along with each other but after checking their site I found a new firmware that fixed a reporting bug. They got along after that, six of them in RAID-5.

 

[AsRock]

Disks should be the same for best performance with raid all so it only takes one to spin out of order to make it fail.

 

[Violet_Shift]

Yeah, the disks are all standard Seagate disks (model ST3000DM001, although the newer one has a different suffix after the model number - this is not the currently misbehaving disk however).

I might nab replacement SATA cables this weekend (just in case) and see if I can swap the questionable drive over to another power cable.

Tried to download HD Sentinel but it keeps failing sadly. I've never noticed any SMART warnings at BIOS or anything though.

SeaTools came up clean.

 

[Disparia]

Yeah, the disks are all standard Seagate disks (model ST3000DM001, although the newer one has a different suffix after the model number - this is not the currently misbehaving disk however).

I might nab replacement SATA cables this weekend (just in case) and see if I can swap the questionable drive over to another power cable.

Tried to download HD Sentinel but it keeps failing sadly. I've never noticed any SMART warnings at BIOS or anything though.

SeaTools came up clean.

http://ask.adaptec.com/app/answers/...with-seagate-barracuda-7200.14-desktop-drives

 

[Violet_Shift]

http://ask.adaptec.com/app/answers/...with-seagate-barracuda-7200.14-desktop-drives

Thanks a lot!

On the EXE version of the firmware updater it says "Don't use on a RAID". Is it safe to use the ISO version on a RAID without wiping all of my data? I'll back up the really important stuff but there's a mountain of stuff I don't have the space elsewhere to keep.

 

[FordGT90Concept]

I wouldn't mess with the firmware without having a backup first. The drives should be removed from the RAID before updating the firmware and then reintroduced to the RAID.

If all of the errors are stemming from one drive, I'd be thinking about replacing it.

You could test the cable by swapping the cable with a non-failing drive and see if the failing follows the cable. Only do this when the RAID is normal. If it degrades on two drives simultaneously, you could risk losing data.

Intel RST can degrade a RAID if the computer shutdown was interrupted. Check the system event logs for warnings/errors, especially those related to an unexpected shutdown.

 

[Disparia]

Thanks a lot!

On the EXE version of the firmware updater it says "Don't use on a RAID". Is it safe to use the ISO version on a RAID without wiping all of my data? I'll back up the really important stuff but there's a mountain of stuff I don't have the space elsewhere to keep.

That's how I did it with the 7200.11s in my other post. Created a bootable CD using the ISO (will also work with USB stick), booted from it, and applied the new firmware to all drives. I probably had to switch from RAID mode to AHCI (or IDE) in the system BIOS for the drives to show up in the updater. After the update need to switch back from AHCI to RAID of course.

I just "did it" because I noticed a problem during the OS install (bad performance) so there was no data to lose if something went wrong. If you can't backup everything you can't live without, work on that issue first.

 

[Violet_Shift]

Oh, I can back up mission critical stuff of course. And I have, and am doing an up to the minute one now as a precaution. I'm more thinking of the "stuff I have that would annoy me to lose" category, such as my music library and my Steam installs (limited download caps are fun).

I'll wait for the rebuild to finish (currently at 80%), shut the PC down, remove the offending drive, plug it into another machine, update firmware, and reintroduce it. If that lasts long enough without any issues I'll repeat the cycle with the remaining 2 drives using the old fw.

 

[Aquinus]

Drives without TLER like the 4x1TB WD Blacks in my RAID-5 paired with having Patrol Read enabled would spontaneously cause my array to degrade on random occasions. If the connection between the south bridge and the CPU is finicky (such as from an overclock,) it might cause an error as well. My RAID does occasionally rebuild/verify my array automatically. I do think RST occasionally confirms that the data is correct (since parity data enables you to not only re-create data but, verify that it's correct,) but it doesn't tend to go to a degraded state when this occurs since it's corrects errors if either the actual data or parity data is correct.

A loose or defective SATA cable could cause issues like this as well. I would suspect this if it's always the same drive causing the error.

 

[Static~Charge]

Tried to download HD Sentinel but it keeps failing sadly. I've never noticed any SMART warnings at BIOS or anything though.

Alternative:

CrystalDiskInfo
http://crystalmark.info/software/CrystalDiskInfo/index-e.html

Download the portable version.
Unzip the file.
Run the appropriate executable: standard (32-bit) or X64.

 

[Violet_Shift]

I ran CrystalDiskInfo and it's saying 3 of the 4 drives are OK. Curiously the drive that's been misbehaving (in the number 4 slot) is OK, and it says the slot 1 disk is "Caution". I don't understand how to read this or if I should be replacing this disk.

 

[FordGT90Concept]

It's the serial number that matters. What does RST say about Z1F0YM7D? Was Z1F0YM7D the one that fell out of the array?

 

[Violet_Shift]

No, it wasn't. Z1F0YM7D is on port 0 according to RST, and it's the port 3 disk that fell out. Although now I'm worried that I have two malfunctioning disks on my hands.

 

[Violet_Shift]

So I just read a few articles saying that the ST3000DM001 is a particularly awful drive with a high fail rate and a bunch of other bad things. Tempted to just clone the array to a single drive from a more reliable provider and run away screaming from Seagate. Glad that I saw this before disaster struck though.

 

[Assimilator]

Heh, if you'd said up front you were using ST3000DM001s I would've told you to run. Glad you figured it out though.

 

[Violet_Shift]

Thanks. If I hadn't came here I wouldn't have figured this out.

I think I'll get a WD 4TB and dump everything on there, and use this RAID until it dies to run games off. Will be curious to see how much longer it lasts after I've flashed the firmware.

 

[AsRock]

Because it's falling out of raid don't mean the drive is failing, it just means it's not keeping in sync with the other drives which as i said you should keep a raid all the same drives then this is less likely to happen.

 

[Violet_Shift]

Hence why I'm not in panic mode yet. I'm backing up important things to drives that are lying around not in use, and I'll replace the cables and flash the fw tomorrow. I'll report back after that.

 

[Aquinus]

I ran CrystalDiskInfo and it's saying 3 of the 4 drives are OK. Curiously the drive that's been misbehaving (in the number 4 slot) is OK, and it says the slot 1 disk is "Caution". I don't understand how to read this or if I should be replacing this disk.View attachment 68286

CrystalDiskInfo does a piss poor job at reading SMART information. Could you use GSmartControl and let us know what it reads for those two fields? If the raw data is simply a hex value, you have 84 uncorrectable sectors, GSmartControl should confirm that. That could be the reason for the RAID becoming degraded.

You can get GSmartControl here.

I highly recommend this tool because it can detect through most RAID setups, properly detects raw values for SMART attributes, and allows you to view and run SMART diagnostic tests.

I personally think that this is the best GUI tool for this task.