• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

TPU Disaster Planning & Recovery

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
27,037 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit

No that's not TPU, I just came across this image while researching and found it funny.

After the recent XS downtime drama I decided to look at our contingency plans in case something bad happens.

  1. A single or multiple files lost from the FS (deleted / corrupted)
  2. Hacker attack, all files deleted on front end
  3. Hacker attack, all DBs deleted
  4. Hacker attack, all files + backup files deleted
  5. Hacker attack, all DB + backup DB deleted
  6. Hacker attack, sensitive information (passwords) leaked
  7. TPU front end web server fail (power, hardware, network)
  8. TPU database server fail (power, hardware, network)
  9. Softlayer Dallas Datacenter destroyed (where TPU DB and WWW servers are)
  10. All Softlayer datacenters destroyed

Please post what else you could imagine can go wrong.
 
Last edited:

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
Geee W1zz is that where all our data is stored??:twitch:
I would suggest backing up all data else where...eg. @ your house
 
Joined
Oct 21, 2008
Messages
3,459 (0.61/day)
Location
95350
Processor ???
Motherboard 760
Cooling HK
Memory 3x1gb
Video Card(s) 285 Classified / EVGA 480
Display(s) 22" Acer /42" Toshiba
Case Stacker
Power Supply Toughpower 1000w / Coolmaster 650w
Software All of them
I guess some one could pee on it and short it out :laugh:
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
or use it as spider webs and act like spider man and pulls all cables out...
 
Joined
Oct 21, 2008
Messages
3,459 (0.61/day)
Location
95350
Processor ???
Motherboard 760
Cooling HK
Memory 3x1gb
Video Card(s) 285 Classified / EVGA 480
Display(s) 22" Acer /42" Toshiba
Case Stacker
Power Supply Toughpower 1000w / Coolmaster 650w
Software All of them
Car comes through the wall and crashes the server :roll::roll::roll:

then you can say some car crashed my server!!:roll:
 

Mussels

Freshwater Moderator
Staff member
Joined
Oct 6, 2004
Messages
58,413 (8.19/day)
Location
Oystralia
System Name Rainbow Sparkles (Power efficient, <350W gaming load)
Processor Ryzen R7 5800x3D (Undervolted, 4.45GHz all core)
Motherboard Asus x570-F (BIOS Modded)
Cooling Alphacool Apex UV - Alphacool Eisblock XPX Aurora + EK Quantum ARGB 3090 w/ active backplate
Memory 2x32GB DDR4 3600 Corsair Vengeance RGB @3866 C18-22-22-22-42 TRFC704 (1.4V Hynix MJR - SoC 1.15V)
Video Card(s) Galax RTX 3090 SG 24GB: Underclocked to 1700Mhz 0.750v (375W down to 250W))
Storage 2TB WD SN850 NVME + 1TB Sasmsung 970 Pro NVME + 1TB Intel 6000P NVME USB 3.2
Display(s) Phillips 32 32M1N5800A (4k144), LG 32" (4K60) | Gigabyte G32QC (2k165) | Phillips 328m6fjrmb (2K144)
Case Fractal Design R6
Audio Device(s) Logitech G560 | Corsair Void pro RGB |Blue Yeti mic
Power Supply Fractal Ion+ 2 860W (Platinum) (This thing is God-tier. Silent and TINY)
Mouse Logitech G Pro wireless + Steelseries Prisma XL
Keyboard Razer Huntsman TE ( Sexy white keycaps)
VR HMD Oculus Rift S + Quest 2
Software Windows 11 pro x64 (Yes, it's genuinely a good OS) OpenRGB - ditch the branded bloatware!
Benchmark Scores Nyooom.
a plane could crash into it. moths in the PSU's.

Some idiot flips the red switch on the PSU's.

All the senior mods die, leaving no one in charge.

replicators from stargate come and eat it.

Skynet takes over all your CPU time.

someone unplugs the cables?
 

DaMulta

My stars went supernova
Joined
Aug 3, 2006
Messages
16,168 (2.50/day)
Location
Oklahoma T-Town
System Name Work in progress
Processor AMD 955---4Ghz
Motherboard MSi GD70
Cooling OcZ Phase/water
Memory Crucial2GB kit (1GBx2), Ballistix 240-pin DIMM, DDR3 PC3-16000
Video Card(s) CrossfireX 2 X HD 4890 1GB OCed to 1000Mhz
Storage SSD 64GB
Display(s) Envision 24'' 1920x1200
Case Using the desk ATM
Audio Device(s) Sucky onboard for now :(
Power Supply 1000W TruePower Quattro
W1zzard sold techPowerUp! for big $$$$. So that he could go live on his island that he is always dreaming, and talking about.
 
Joined
Oct 21, 2008
Messages
3,459 (0.61/day)
Location
95350
Processor ???
Motherboard 760
Cooling HK
Memory 3x1gb
Video Card(s) 285 Classified / EVGA 480
Display(s) 22" Acer /42" Toshiba
Case Stacker
Power Supply Toughpower 1000w / Coolmaster 650w
Software All of them
or
You get downgraded to dial up!
you said hacker 4 times and you didnt say ninja!

Chicken_Pattys ninjas are every were! watch your back!
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
a nuke?
an enemy?
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
27,037 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit
PLEASE look at the list I posted, everything you suggested so far is covered under those.
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
earthquake and the servers fall causing physical damage?
Extreme flooding?(not sure if these servers are underground)
 

DaMulta

My stars went supernova
Joined
Aug 3, 2006
Messages
16,168 (2.50/day)
Location
Oklahoma T-Town
System Name Work in progress
Processor AMD 955---4Ghz
Motherboard MSi GD70
Cooling OcZ Phase/water
Memory Crucial2GB kit (1GBx2), Ballistix 240-pin DIMM, DDR3 PC3-16000
Video Card(s) CrossfireX 2 X HD 4890 1GB OCed to 1000Mhz
Storage SSD 64GB
Display(s) Envision 24'' 1920x1200
Case Using the desk ATM
Audio Device(s) Sucky onboard for now :(
Power Supply 1000W TruePower Quattro
A Hurricane hits Dallas and knocks everything offline.

BP Oil was really scared of this happening last year.
 

DaMulta

My stars went supernova
Joined
Aug 3, 2006
Messages
16,168 (2.50/day)
Location
Oklahoma T-Town
System Name Work in progress
Processor AMD 955---4Ghz
Motherboard MSi GD70
Cooling OcZ Phase/water
Memory Crucial2GB kit (1GBx2), Ballistix 240-pin DIMM, DDR3 PC3-16000
Video Card(s) CrossfireX 2 X HD 4890 1GB OCed to 1000Mhz
Storage SSD 64GB
Display(s) Envision 24'' 1920x1200
Case Using the desk ATM
Audio Device(s) Sucky onboard for now :(
Power Supply 1000W TruePower Quattro
My dad is a Tree Network Support Engineer(I guess you could call it that, they call him when anything fails across the globe). If you would like I could ask him if he could make a real list of things that could happen....
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
27,037 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit
My dad is a Tree Network Support Engineer(I guess you could call it that, they call him when anything fails across the globe). If you would like I could ask him if he could make a real list of things that could happen....

would appreciate that. a high-level list is fine.

something like "datacenter destroyed" covers "earthquake, flood, tornado, terrorists, volcano"
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
ahh sorry:p
terrirost attack? again datacentre destroyed...
 
Joined
Jan 11, 2009
Messages
9,232 (1.66/day)
Location
Montreal, Canada
System Name Homelabs
Processor Ryzen 5900x | Ryzen 1920X
Motherboard Asus ProArt x570 Creator | AsRock X399 fatal1ty gaming
Cooling Silent Loop 2 280mm | Dark Rock Pro TR4
Memory 128GB (4x32gb) DDR4 3600Mhz | 128GB (8x16GB) DDR4 2933Mhz
Video Card(s) EVGA RTX 3080 | ASUS Strix GTX 970
Storage Optane 900p + NVMe | Optane 900p + 8TB SATA SSDs + 48TB HDDs
Display(s) Alienware AW3423dw QD-OLED | HP Omen 32 1440p
Case be quiet! Dark Base Pro 900 rev 2 | be quiet! Silent Base 800
Power Supply Corsair RM750x + sleeved cables| EVGA P2 750W
Mouse Razer Viper Ultimate (still has buttons on the right side, crucial as I'm a southpaw)
Keyboard Razer Huntsman Elite, Pro Type | Logitech G915 TKL
A beaver (or any other animal) eats the cables o_O
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
27,037 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit
A beaver (or any other animal) eats the cables o_O

no they won't let small animals without security badge into the datacentre
 

h3llb3nd4

New Member
Joined
Feb 15, 2009
Messages
3,323 (0.60/day)
Location
Durban, South Africa
System Name My mobo is Laughing at me
Processor E7400
Motherboard P5KPL-E Bios flashed to 0601 (Piece of poo!!)
Cooling Thermalright Ultra 120a
Memory GENERIC 2 GB DDR2 800
Video Card(s) NONE!! Beat that!!
Storage 500GB SAMSUNG SATAII, 250GB SAMSUNG SATAII and o'l crappy 4gb maxtor
Display(s) ACER X223W Q
Case AEROCOOL ZERODEGREE (planning to mod)
Audio Device(s) REALTEK ONBOARD
Power Supply GIGABYTE 460W
Software Win 7 x86 build 7022
Benchmark Scores Super Pi 1m 17.000 :(
a mouse? you can't catch that easily...
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
27,037 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit
a mouse? you can't catch that easily...

only when connected to computers. belongs into "server fail (power, hardware, network)" anyway
 
Joined
Jun 28, 2008
Messages
1,668 (0.29/day)
Location
Newcastle
Processor Intel Core i5 3570K
Motherboard MSI H61m-P31/w8
Cooling Stock
Memory 2*8GB DDR3
Video Card(s) XFX HD7850
Case Coolermaster Elite 330
Audio Device(s) Realtek HD audio
Power Supply Corsair RM650
Software Windows 10
Big ass hurricane cutting off the internet or power supply to the data center. Some type of disease spreads like wildfire through Dallas killing all the workers at the data center.
Someone cuts some of the MASSIVE fibre optic cables under the sea so the mod team can't reach it, except for 2 or 3 of the US ones.
 

ShadowFold

New Member
Joined
Dec 23, 2007
Messages
16,918 (2.84/day)
Location
Omaha, NE
System Name The ShadowFold Draconis (Ordering soon)
Processor AMD Phenom II X6 1055T 2.8ghz
Motherboard ASUS M4A87TD EVO AM3 AMD 870
Cooling Stock
Memory Kingston ValueRAM 4GB DDR3-1333
Video Card(s) XFX ATi Radeon HD 5850 1gb
Storage Western Digital 640gb
Display(s) Acer 21.5" 5ms Full HD 1920x1080P
Case Antec Nine-Hundred
Audio Device(s) Onboard + Creative "Fatal1ty" Headset
Power Supply Antec Earthwatts 650w
Software Windows 7 Home Premium 64bit
Benchmark Scores -❶-❸-❸-❼-
What if Intel paid you to move the servers to their SSD's?
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
27,037 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit

DaMulta

My stars went supernova
Joined
Aug 3, 2006
Messages
16,168 (2.50/day)
Location
Oklahoma T-Town
System Name Work in progress
Processor AMD 955---4Ghz
Motherboard MSi GD70
Cooling OcZ Phase/water
Memory Crucial2GB kit (1GBx2), Ballistix 240-pin DIMM, DDR3 PC3-16000
Video Card(s) CrossfireX 2 X HD 4890 1GB OCed to 1000Mhz
Storage SSD 64GB
Display(s) Envision 24'' 1920x1200
Case Using the desk ATM
Audio Device(s) Sucky onboard for now :(
Power Supply 1000W TruePower Quattro
would appreciate that. a high-level list is fine.

something like "datacenter destroyed" covers "earthquake, flood, tornado, terrorists, volcano"

ROFL a mirror redundant system in another location for solve all of those issues:toast:
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
27,037 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit
ROFL a mirror redundant system in another location for solve all of those issues:toast:

yes, i'm looking for a list of issues now. then i will check what is doable with a reasonable budget
 

Mussels

Freshwater Moderator
Staff member
Joined
Oct 6, 2004
Messages
58,413 (8.19/day)
Location
Oystralia
System Name Rainbow Sparkles (Power efficient, <350W gaming load)
Processor Ryzen R7 5800x3D (Undervolted, 4.45GHz all core)
Motherboard Asus x570-F (BIOS Modded)
Cooling Alphacool Apex UV - Alphacool Eisblock XPX Aurora + EK Quantum ARGB 3090 w/ active backplate
Memory 2x32GB DDR4 3600 Corsair Vengeance RGB @3866 C18-22-22-22-42 TRFC704 (1.4V Hynix MJR - SoC 1.15V)
Video Card(s) Galax RTX 3090 SG 24GB: Underclocked to 1700Mhz 0.750v (375W down to 250W))
Storage 2TB WD SN850 NVME + 1TB Sasmsung 970 Pro NVME + 1TB Intel 6000P NVME USB 3.2
Display(s) Phillips 32 32M1N5800A (4k144), LG 32" (4K60) | Gigabyte G32QC (2k165) | Phillips 328m6fjrmb (2K144)
Case Fractal Design R6
Audio Device(s) Logitech G560 | Corsair Void pro RGB |Blue Yeti mic
Power Supply Fractal Ion+ 2 860W (Platinum) (This thing is God-tier. Silent and TINY)
Mouse Logitech G Pro wireless + Steelseries Prisma XL
Keyboard Razer Huntsman TE ( Sexy white keycaps)
VR HMD Oculus Rift S + Quest 2
Software Windows 11 pro x64 (Yes, it's genuinely a good OS) OpenRGB - ditch the branded bloatware!
Benchmark Scores Nyooom.
would appreciate that. a high-level list is fine.

something like "datacenter destroyed" covers "earthquake, flood, tornado, terrorists, volcano"

the datacenter doesnt pay its electric bill and it gets turned off.

XS staff sneak in and OC it and it becomes unstable...
 
Top