How Much Should You Back Up Your Data and What is the Likelihood of Drive Failure?

[avrona]

So I've been doing some research into how much people backup their data and what the odds are of their backups actually coming in handy, as I've never really understood the whole backup culture that exists in the tech world. I have no backups of any of my files and thankfully I've never needed one, so I decided to do some research and make a video on the subject. So I just want to know, if you are aware of the percentages and likelihoods of different types of drives failing, please do let me know, and also let me know if you backup your data, if you do, what kind of data is it, and how much do you recommend both normal PC users and more advance users backup their data? And what do you think of using RAID 1 or RAID 10 as a type of backup?

 

[sneekypeet]

All it takes is that one time a drive fails full of information you need, and you will become aware of the importance of a backup. I'm not sure percentages really matter, as any percent, when it happens to you, could spell big trouble. For the average users, with no important data needed, if things go belly-up, you can have a fresh OS in a matter of hours.

Personally, I work from a PC, and without backups of important data, I could lose everything in an instant. Normally I back everything up to my NAS once a month (1st backup), and at the same time I back up to a HDD that does not stay in the system powered on all of the time (2nd backup). The one thing I need to do on top of this is to have another backup off site. I know it may be a bit of a stretch, but if I were to have a fire or a flood, all of my data could still be lost

 

[Ahhzz]

You're not going to be able to get any hard numbers on percentages of failure, for a multitude of explanations. Newer technology (NVMe Drives) show a lower failure rate than SATA drives for many reasons, but primary among those is that they've not been in widespread use for nearly as long as SATA drives, and so we don't have a good idea of how long they'll really last. SATA drives, according to Wiki, were announced in 2000, and starting going into serious use around 2003. NVMe were announced in 2011, and don't appear to be getting installed for at least another 3-4 years, in large numbers.

Realistically speaking, the use of Solid State Drives like the original SSDs and the newer NVMe drives should experience lower failure rates, and thus you shouldn't need a backup as badly these days as in the days of SATA and *shudder* IDE drives. This is primarily due to being more durable and resistant to common failure issues (temperature, dust in the environment, physical shock, etc).

I commonly recommend my serious users to use the 3-2-1 rule. 3 backups, 2 different types of storage media, and 1 of those offsite. For home users with pictures they cherish, I push 2 externals, and at least 1 cloud service, altho I am prone to recommending 2-3 different clouds for "Just in Case".

 

[avrona]

You're not going to be able to get any hard numbers on percentages of failure, for a multitude of explanations. Newer technology (NVMe Drives) show a lower failure rate than SATA drives for many reasons, but primary among those is that they've not been in widespread use for nearly as long as SATA drives, and so we don't have a good idea of how long they'll really last. SATA drives, according to Wiki, were announced in 2000, and starting going into serious use around 2003. NVMe were announced in 2011, and don't appear to be getting installed for at least another 3-4 years, in large numbers.

Realistically speaking, the use of Solid State Drives like the original SSDs and the newer NVMe drives should experience lower failure rates, and thus you shouldn't need a backup as badly these days as in the days of SATA and *shudder* IDE drives. This is primarily due to being more durable and resistant to common failure issues (temperature, dust in the environment, physical shock, etc).

I commonly recommend my serious users to use the 3-2-1 rule. 3 backups, 2 different types of storage media, and 1 of those offsite. For home users with pictures they cherish, I push 2 externals, and at least 1 cloud service, altho I am prone to recommending 2-3 different clouds for "Just in Case".

What do you count as serious users and why would they need so many backups? Also with home users, what would be the use for 2 or more external drives? I have plenty of pictures I cherish, and a whole bunch of channel related files, yet I've never seen any use for a backup. Should I do one?

 

[sneekypeet]

What do you count as serious users and why would they need so many backups? Also with home users, what would be the use for 2 or more external drives? I have plenty of pictures I cherish, and a whole bunch of channel related files, yet I've never seen any use for a backup. Should I do one?

If you want to keep them, yes!
If you were to run into a natural disaster, or even if the drive just went belly up, without a backup, they are all gone.

 

[Ahhzz]

What do you count as serious users and why would they need so many backups? Also with home users, what would be the use for 2 or more external drives? I have plenty of pictures I cherish, and a whole bunch of channel related files, yet I've never seen any use for a backup. Should I do one?

Serious users, for me, would include clients such as: Accountants, Attorneys, medical practices (although these are touchy due to privacy), businesses which could be subject to lawsuits (construction, chemicals, etc). The list is pretty long, and includes most of our clients. If you'll check the link on the 3-2-1, that will explain more than adequately why they recommend that, and their logic is sound for our purposes.
For home users with pictures they cherish, a computer is way too subject to failure to not have backups of some sort. Cloud works well for most of them, and at least an external.

To summarize, offsite is in case your house burns down. Onsite external is in case your computer burns down. If you've forgotten your online password, or the cloud company goes defunct overnight (or while you weren't looking), you don't lose everything. I've used the line "Ok, I'm a disgruntled employee, and I just tossed your PC out the 10-story window into the pond out there. What can't you lose?" That usually gets a couple of chuckles, more horrified looks, and then an order for the quickest backup we can order.

 

[EarthDog]

If you want to minimize the risk and not lose those cherished pictures/files, you will be doing a backup. Eventually, EVERY HDD/SSD etc, will fail. Be it within warranty (less likely) or out (more likely).

As far as failures, the best we have is information from a builder or backblaze (though this is data center loads and not realistic compared to home use) on part failure. Overall it is low, but the risk is there. Not having a backup is a recipe for losing whatever it is you have on the drives.

Most Reliable PC Hardware of 2018

At Puget Systems, one of the most important things we track in our workstations is the failure rates of individual components. Overall, 2018 was a very good year for hardware reliability with about half as many parts failing this year versus 2015, 2016, or 2017. But what models were the best of...

www.pugetsystems.com

(this information is from within the first year, note... failures obviously can happen and at a higher rate on older drives.

The need for backups in a Enterprise environment you understand, right?

Please tell me your video is on the merits of taking a backup and not your single anecdotal experience of not needing one.... if your followers have half a clue, you'll get skewered in comments.

EDIT: To answer the questions directly...

1. I'm well aware of the percentages of failures. Google is my friend!
2. Of course I back up my data.
2a. It contains, pictures, MS Word/Excel files for personal and work use. I also have a base backup of my OS. Like Day 0 with just chipset and GPU drivers, as well as a few apps I use frequently. I also backup my games drive as well. This allows me to restore from a HDD which is much faster than even my 400 Mbps internet to DL and then install. I can re-associate steam games for example.
3. For data, it depends on how frequently it is changed (you can also do incremental backups). For a normal enthusiast, I recommend taking a monthly backup. For those with more critical information, perhaps weekly or even daily.

 

[John Naylor]

There are 2 types of people in this world:

a) Those that have had a storage drive failure.
b) Thos who have not yet had a storage drive failure.

% of HD failures are (well were) published. Not the usual nonsense that Backblaze publishes where drives are installed contrary to manufacturer's specifications.... but real hard data.

Disques durs - Les taux de retour des composants (15) - HardWare.fr

La fiabilité est aussi importante que difficile à mesurer, si bien qu'elle reste inconnue. Cartes mères et graphiques, alims, ram, hdd et ssd : voici les chiffres dont nous disposons !

www.hardware.fr

• Seagate 0,72% (contre 0,69%)
• Toshiba 0,80% (contre 1,15%)
• Western 1,04% (contre 1,03%)
• HGST 1,13% (contre 0,60%)

The 1st number was previous 6 months and 2nd was thr 6 months preceding that.

It's not the relative difference is RMA rates between brands that is significant ... its the specific models. I have seen drives with as high as 10%. In general, large drives fail more often. In SSDs 1st few years, they had a pretty high failure rate. In the last 8 years... we have had

(4) SSD Failures
(0) SSHD failures
(1) HD Failure

We stopped using HDs 7 years ago ... every box has a at least (1) SSD and (1) SSHD

How often you back up depends upon how valuable your data is:

a) gaming box ... no big loss , replace drive, redownload your games
b) 20 years of business records ... kinda important
c) 25 years worth of CAD drawings and project specs ... critical importance.

We have a network storage drive. It's mirrored to a 2nd drive in same box. If it dies, switch cable and we are up and running again. A 3rd drive us used in a d pocking staion and SSHD is stored off site.

It must be said that as you "don't understand why people do backups", you're really not well suited to prepare a video on the subject.

 

[Solaris17]

Also with home users, what would be the use for 2 or more external drives? I have plenty of pictures I cherish, and a whole bunch of channel related files, yet I've never seen any use for a backup. Should I do one?

Iv had clients that have lost data. The one that sticks with me is the guy that came in with a PC that wouldnt boot. Begged me to save his pictures. His daughter died in the single digits after getting sick. His wife a few years prior in a car accident. His only pictures of his dead daughter and wife for this "normal home user" were on this desktop in which the most complicated thing he did was check gmail.

Iv had customer lose tons of data but I always remember this one.

If you use statistics to decide the worth of those pictures I feel sorry for the black and white way in which you look at life.

 

[avrona]

Please tell me your video is on the merits of taking a backup and not your single anecdotal experience of not needing one.... if your followers have half a clue, you'll get skewered in comments.

Actually kinda neither, the video will be me talking about how I think backups aren't really needed in most cases, and I will be mainly tackling the 3,2,1 rule and the amount of backups a normal user should have, along with showing how unlikely it is that a backup could help, at least that's the plan based on my current opinions and the data I gathered so far, as the point of this thread was to get some more insights on the topic so probably some of my opinions on it will change before I make the full video.

 

[EarthDog]

Actually kinda neither, the video will be me talking about how I think backups aren't really needed in most cases, and I will be mainly tackling the 3,2,1 rule and the amount of backups a normal user should have, along with showing how unlikely it is that a backup could help, at least that's the plan based on my current opinions and the data I gathered so far, as the point of this thread was to get some more insights on the topic so probably some of my opinions on it will change before I make the full video.

3/2/1 rule is more or less Enterprise to me. I mean, that IS a proper way to do it mitigating any of the 'most likely' of threats, but it is overkill for many, especially home users. I worked in a data center in many capacities (operations, management, capacity planning, etc)... for our backups (this is a water company mind you). We kept the active copy on the device, another copy in a different part of the building (mini data center for backup storage only), as well as offsite storage (tapes, LTO4) we cycled through. This way if the data center and the device bit the dust, we have another copy. If the building went up in smoke, we had a copy offsite to restore business. It's all about risk management.

My man, if this is YT vid, you are going to get skewered following your logic. You'll find your thought process is really in the minority.

Just because you haven't lost something valuable, doesn't mean there isn't a lot of value in backing what you/user deems to be critical for them.

 

[ERazer]

Family photos/documents backed up in NAS then cloud.
Media files/steam library to NAS.
In NAS i can have up to 2 drive fails and still able to rebuild.

 

[avrona]

3/2/1 rule is more or less Enterprise to me. I mean, that IS a proper way to do it mitigating any of the 'most likely' of threats, but it is overkill for many, especially home users. I worked in a data center in many capacities (operations, management, capacity planning, etc)... for our backups (this is a water company mind you). We kept the active copy on the device, another copy in a different part of the building (mini data center for backup storage only), as well as offsite storage (tapes, LTO4) we cycled through. This way if the data center and the device bit the dust, we have another copy. If the building went up in smoke, we had a copy offsite to restore business. It's all about risk management.

My man, if this is YT vid, you are going to get skewered following your logic. You'll find your thought process is really in the minority.

Just because you haven't lost something valuable, doesn't mean there isn't a lot of value in backing what you/user deems to be critical for them.

I know it is in the minority, and I honestly I don't care if I get "skewered in the comments" as there is logic to my logic on this topic, and I am going to try to explain my logic to the best of my ability.

 

[Vayra86]

What do you count as serious users and why would they need so many backups? Also with home users, what would be the use for 2 or more external drives? I have plenty of pictures I cherish, and a whole bunch of channel related files, yet I've never seen any use for a backup. Should I do one?

Definitely. Its not hard to have a thumb drive laying around somewhere. Totally worth doing. If its about your pictures (from holidays, events etc.) its just a small adjustment, when you copy them over to your rig, to also copy them to a USB stick in one go. If you'd do that every time you return from holiday, you've also got a nice frequency to your backups right away.

And you can do the same thing for your Documents folder, for example. Just copy paste over the last one, and done. There's no need to overcomplicate it with automation or anything.

You can then take it one step further and put one thumb drive per X years offsite. Such as with a family member.

Another good option these days is just using the cloud backup. That is something you can easily automate. Have OneDrive sort it out for you, done - you have an offsite backup. And if you take your pictures with your phone, its even easier.

Iv had clients that have lost data. The one that sticks with me is the guy that came in with a PC that wouldnt boot. Begged me to save his pictures. His daughter died in the single digits after getting sick. His wife a few years prior in a car accident. His only pictures of his dead daughter and wife for this "normal home user" were on this desktop in which the most complicated thing he did was check gmail.

Iv had customer lose tons of data but I always remember this one.

If you use statistics to decide the worth of those pictures I feel sorry for the black and white way in which you look at life.

If I would lose the pics of my daughter's birth etc.... man. That'd be bad. Imagine having to even explain that later on 'Yeah, I didn't think we needed a backup'. My wife already lost a bunch of her pictures from her 20's, its a few years ago but she still mentions it every once in a while. Its like having something stolen from you.

 

[EarthDog]

I know it is in the minority, and I honestly I don't care if I get "skewered in the comments" as there is logic to my logic on this topic, and I am going to try to explain my logic to the best of my ability.

I'm waiting with bated breath to hear this 'logic on logic'... But, I could easily be missing something.

 

[neatfeatguy]

I have multiple backups of important files on multiple types of media.

Any important files I keep on USB and DVD, along with an external HDD. I keep a mirror image of my SSD, I update this once a month and that might even be too often since I don't keep much on it aside from the OS and important system programs such as antivirus/malware and one or two games that I frequent. I don't really add much of anything to it. 99.9% of all new data I do utilize is to my spin drives on my PC and my Plex server.

All family pictures/videos are copied in triplicate across the 2 HDDs in my PC and one HDD on the Plex server.

When I first cobbled together my Plex server, I had the two 4TB drivers setup in RAID1. Then Windows 10 updated (I think it was the Fall Creator update) and broke itself so bad neither drive would boot into Windows. Since windows wouldn't boot I tried Safe Mode - wouldn't load. Since Safe Mode wasn't accessible I tried Win 10 Recovery option - this didn't work, Windows couldn't recover. So, I was left with the option of format and re-install. Now I just have the data backup from the first drive to the second drive and I don't keep them in RAID1.

I always like to keep backups. Once the data is gone, it's gone. You won't get it back. If you don't have backups and you have a catastrophic failure, that's it. I've seen multiple business owners lose all their data history because they didn't keep any kind of backup. Even after I spoke to them about something as simple as an external drive to copy the files to.....they'd call in with a panic in their voice that all their info was gone because their computer crashed. Most of the time we could access the bad drive and recover some data, but that wasn't always the case. So I've learned to keep data you find important, backed up.

 

[Ahhzz]

There are 2 types of people in this world:

a) Those that have had a storage drive failure.
b) Thos who have not yet had a storage drive failure.

% of HD failures are (well were) published. Not the usual nonsense that Backblaze publishes where drives are installed contrary to manufacturer's specifications.... but real hard data.

Disques durs - Les taux de retour des composants (15) - HardWare.fr

La fiabilité est aussi importante que difficile à mesurer, si bien qu'elle reste inconnue. Cartes mères et graphiques, alims, ram, hdd et ssd : voici les chiffres dont nous disposons !

www.hardware.fr

• Seagate 0,72% (contre 0,69%)
• Toshiba 0,80% (contre 1,15%)
• Western 1,04% (contre 1,03%)
• HGST 1,13% (contre 0,60%)

The 1st number was previous 6 months and 2nd was thr 6 months preceding that.

It's not the relative difference is RMA rates between brands that is significant ... its the specific models. I have seen drives with as high as 10%. In general, large drives fail more often. In SSDs 1st few years, they had a pretty high failure rate. In the last 8 years... we have had

(4) SSD Failures
(0) SSHD failures
(1) HD Failure

We stopped using HDs 7 years ago ... every box has a at least (1) SSD and (1) SSHD

How often you back up depends upon how valuable your data is:

a) gaming box ... no big loss , replace drive, redownload your games
b) 20 years of business records ... kinda important
c) 25 years worth of CAD drawings and project specs ... critical importance.

We have a network storage drive. It's mirrored to a 2nd drive in same box. If it dies, switch cable and we are up and running again. A 3rd drive us used in a d pocking staion and SSHD is stored off site.

It must be said that as you "don't understand why people do backups", you're really not well suited to prepare a video on the subject.

Interesting data from 3 years ago, however not very representative, any more than any other set of statistics regarding hard drives. From the article:
"It should be added that these statistics are logically limited to the products sold by this e-merchant, and to customer returns made directly via this one, which is not always the case since it is possible to make returns directly. with the manufacturer, especially in the field of storage: however this represents a minority in the first year, especially since the e-merchant from which these statistics come out bears the cost of return. "

As noted, Backblaze doesn't use the drives they report data on specifically as directed by the hard drive manufacturers, since they use them in data storage centers. This means they get more constant usage, as well as high continuous run time. It is nice to note from their data, however, which drives manage to maintain a high pass rate, as it could be taken to indicate reliable drives for a long period of time.

To answer your basic questions, Yes, I use backups for the data critical to me, or (as data is cheap) difficult to replace: tax documentation, resume, scans of certain documents, and photos. I use cloud storages (plural), external USB drives, a file server local to my house, and a Netgear NAS under RAID 1.

The discussion regarding Raid 1 and Raid 10 vs a "real" backup brings the definition of "Backup" in play. By definition, a Raid array is not a backup. However, for many people, myself included, having a additional copy of my data identical to the first available in case of catastrophic drive failure counts as backup.

 

[authorized]

Percentages don't really matter, all you need to realize that it can happen, at any moment (and happens everyday to someone). What really matters is how important your data is.
Proper backup is kind of like insurance, it has some cost and inconvenience associated with it and most of the time, possibly never, pays off. But it's there just in case, protecting you from getting truly screwed, from worst case scenario.

Also, RAID is not backup.

 

[avrona]

Percentages don't really matter, all you need to realize that it can happen, at any moment (and happens everyday to someone). What really matters is how important your data is.
Proper backup is kind of like insurance, it has some cost and inconvenience associated with it and most of the time, possibly never, pays off. But it's there just in case, protecting you from getting truly screwed, from worst case scenario.

Also, RAID is not backup.

So many other horrible things can also happen to us, yet the chances of those things happening are as rare as drive fails, so what's the difference? Also why exactly is RAID not a backup?

 

[sneekypeet]

Also why exactly is RAID not a backup?

More importantly, some types of RAID are not a backup.

 

[avrona]

More importantly, some types of RAID are not a backup.

Yes I know that RAID 0 is not certainly a backup, but 1 is.

 

[Ahhzz]

Yes I know that RAID 0 is not certainly a backup, but 1 is.

The reason that RAID is not considered a technical "backup" is because a backup is supposed to protect you from loss of data due to events such as drive failure (obviously), loss of hardware (fire, theft, etc), accidental or malicious deletion (disgruntled employee/host manager) or massive data corruption (ion storm? lol). There are other potential causes of data loss, but already we've run into 2 situations where a RAID does not provide backup protection: Deletion or Data Corruption. A "mirrored" raid of any sort, by definition, is an exact copy of the data onto multiple drives. Let me repeat that: "Exact copy of the data". If the data is deleted, a RAID does exactly what it's supposed to: Deletes The Data. Likewise, a file which becomes corrupted becomes a duplicated corrupted file. Even if somehow the RAID doesn't have the ability to copy the corruption, that RAID is not programmed to kow the different between intentional or accidental bit-flipping.
Therefore, RAID is not an accepted form of "Backup" by the definition.

 

[Toothless]

I'll just say I have a backup of my documents and a backup of that backup. Came in handy a few or more months ago when that drive completely failed.

 

[Deleted member 158293]

2 on site (raid not considered a backup), 2 off site backups.

Might take a few months or a decade+... Experiencing the first time getting caught without a backup is the best teacher.

 

[Ahhzz]

I'll just say I have a backup of my documents and a backup of that backup. Came in handy a few or more months ago when that drive completely failed.

several years ago, I splurged on Hard Disk Sentinel. Reasoning was that a few months earlier, I had seen an error on restart on my file server about a corrupted file and disk scan. I dug thru the Event Viewer, but got misled because of the way I had initially loaded the system, and started keeping a "Close Eye" on one of my older data drives. Fast forward 2 months or so, and my System drive (a Raptor 10k) crashed its board, taking my tax data, loads of email, and other random files I'll probably never know were gone. Fortunately, most of my critical data like all my photos, were stored in the data drives and on an HP MediaVault, but I seriously considered paying to get it rebuilt.

I've actually got an eBay page up at home right now, thinking about buying a replacement board to see if I can resurrect it, but it keeps referring to something like flashing the bios on the circuit board, and I'm a little concerned I won't have the components to do it heheh.

Too many people think "Ah, I've never had a data loss, who needs a backup??". Those people make really good customers down the road. "Can you recover my data?? What can I do to keep it from happening again?? Can you schedule annual checks on my drives???!"