• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Cloud computing gone bad? Amazon servers down...

johnspack

Here For Good!
Joined
Oct 6, 2007
Messages
5,979 (0.99/day)
Location
Nelson B.C. Canada
System Name System2 Blacknet , System1 Blacknet2
Processor System2 Threadripper 1920x, System1 2699 v3
Motherboard System2 Asrock Fatality x399 Professional Gaming, System1 Asus X99-A
Cooling System2 Noctua NH-U14 TR4-SP3 Dual 140mm fans, System1 AIO
Memory System2 64GBS DDR4 3000, System1 32gbs DDR4 2400
Video Card(s) System2 GTX 980Ti System1 GTX 970
Storage System2 4x SSDs + NVme= 2.250TB 2xStorage Drives=8TB System1 3x SSDs=2TB
Display(s) 2x 24" 1080 displays
Case System2 Some Nzxt case with soundproofing...
Audio Device(s) Asus Xonar U7 MKII
Power Supply System2 EVGA 750 Watt, System1 XFX XTR 750 Watt
Mouse Logitech G900 Chaos Spectrum
Keyboard Ducky
Software Manjaro, Windows 10, Kubuntu 23.10
Benchmark Scores It's linux baby!

Kreij

Senior Monkey Moderator
Joined
Feb 6, 2007
Messages
13,817 (2.21/day)
Location
Cheeseland (Wisconsin, USA)
Personally, I think the "cloud" is a disaster just waiting to happen.
 
Joined
Feb 18, 2010
Messages
1,850 (0.36/day)
System Name Eldritch
Processor AMD Ryzen 5 5800X3D
Motherboard ASUS TUF X570 Pro Wifi
Cooling Satan's butthole after going to Taco Bell
Memory 64 GB G.Skill TridentZ
Video Card(s) Vega 56
Storage 6*8TB Western Digital Blues in RAID 6, 2*512 GB Samsung 960 Pros
Display(s) Acer CB281HK
Case Phanteks Enthoo Pro PH-ES614P_BK
Audio Device(s) ASUS Xonar DX
Power Supply EVGA Supernova 750 G2
Mouse Razer Viper 8K
Software Debian Bullseye
Everybody! Put all your eggs in one basket that likely never gets updated!
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
Same. The only reason why businesses like Google, Amazon, and Microsoft are pushing it is because it gives them information about you and information is power. I like to keep all my data local, secure, and with multiple backups, thank you very much.
 

streetfighter 2

New Member
Joined
Jul 26, 2010
Messages
1,655 (0.33/day)
Location
Philly
What's "ROF sim"?
Personally, I think the "cloud" is a disaster just waiting to happen.
It's always sounded that way to me . . . On the other hand I was under the impression that the "cloud" is supposed to negate SPOF with various failsafe mechanisms (such as failover).

I'm disappointed that EC2 went down like dominoes.
 

Kreij

Senior Monkey Moderator
Joined
Feb 6, 2007
Messages
13,817 (2.21/day)
Location
Cheeseland (Wisconsin, USA)
I guess my question in this case would be, "Where is the "cloud" redundancy?"
If Amazon's Virginia site went down, why wasn't all traffic immediately rerouted to another cloud location where the data was replicated in real time?

It's not being replicated? Nice, that's network management 101 stuff. :shadedshu

If I was a cloud customer who is paying for terabytes of data to be available to my customers and it became unavailable, I would be .. well .. perturbed.

@SF2 : ROF is Rise Of Flight.
 
Last edited:

johnspack

Here For Good!
Joined
Oct 6, 2007
Messages
5,979 (0.99/day)
Location
Nelson B.C. Canada
System Name System2 Blacknet , System1 Blacknet2
Processor System2 Threadripper 1920x, System1 2699 v3
Motherboard System2 Asrock Fatality x399 Professional Gaming, System1 Asus X99-A
Cooling System2 Noctua NH-U14 TR4-SP3 Dual 140mm fans, System1 AIO
Memory System2 64GBS DDR4 3000, System1 32gbs DDR4 2400
Video Card(s) System2 GTX 980Ti System1 GTX 970
Storage System2 4x SSDs + NVme= 2.250TB 2xStorage Drives=8TB System1 3x SSDs=2TB
Display(s) 2x 24" 1080 displays
Case System2 Some Nzxt case with soundproofing...
Audio Device(s) Asus Xonar U7 MKII
Power Supply System2 EVGA 750 Watt, System1 XFX XTR 750 Watt
Mouse Logitech G900 Chaos Spectrum
Keyboard Ducky
Software Manjaro, Windows 10, Kubuntu 23.10
Benchmark Scores It's linux baby!
Man, it's still down. I just bought a plane, and someone gifted me one, which I haven't even seen yet. We are trying to finance the next major patch to our sim engine. This hacker crap has just destroyed $1000s of effort on our part. What crap.
 
Joined
Jun 13, 2009
Messages
1,539 (0.28/day)
Location
Canada/Québec/Montreal
System Name Main PC
Processor PII 925 x4 @3.724GHz (266x14) 1.525v NB 2660 1.425v
Motherboard Gigabyte AM3 GA-890XA-UD3 (790x+SB850)
Cooling Scythe Mugen 2 rev.B
Memory Hyperx 8GB (2x4) 1600@1418 8-7-7-20-27-1t
Video Card(s) GTX 680
Storage 256GB SSD / 2TB HDD
Display(s) LCD Samsung 24" 16:9
Case Cooler Master HAF 912
Audio Device(s) On-Board HD
Power Supply CM 750w GX |3.3v@25a|5v@25a|12v@60a
Software Kubuntu dual boot /Windows 7 Ultimate 64bit
Benchmark Scores later...

Funtoss

New Member
Joined
Mar 4, 2011
Messages
588 (0.12/day)
Location
New Zealand
System Name My Heater
Processor Intel Core 2 Quad Q6600 @ 3.2ghz
Motherboard Asus P5B-E Plus
Cooling Zalman CNPS 9700 NT
Memory Patriot 4GB 800MHz DDR2
Video Card(s) EVGA GTX 460 1GB
Storage 2x Western Digital 640GB (1.28TB)
Display(s) Viewsonic VX2349wm 24"
Case Raidmax Hurricane
Audio Device(s) C-Media 8738
Power Supply Aywun Megapower 650w 80+
Software Windows 7 64 Bit
what is cloud computing? o_O
 
Joined
Mar 26, 2010
Messages
9,762 (1.91/day)
Location
Jakarta, Indonesia
System Name micropage7
Processor Intel Xeon X3470
Motherboard Gigabyte Technology Co. Ltd. P55A-UD3R (Socket 1156)
Cooling Enermax ETS-T40F
Memory Samsung 8.00GB Dual-Channel DDR3
Video Card(s) NVIDIA Quadro FX 1800
Storage V-GEN03AS18EU120GB, Seagate 2 x 1TB and Seagate 4TB
Display(s) Samsung 21 inch LCD Wide Screen
Case Icute Super 18
Audio Device(s) Auzentech X-Fi Forte
Power Supply Silverstone 600 Watt
Mouse Logitech G502
Keyboard Sades Excalibur + Taihao keycaps
Software Win 7 64-bit
Benchmark Scores Classified
what is cloud computing? o_O

please read this

cloud is pretty nice concept that said you can access your data no matter where you are and what device you are on
but for today i guess theres alot that need to do from traffic, apps, security and so
maybe in the future we could, but for today i guess its still need a long way to go
 

Easy Rhino

Linux Advocate
Staff member
Joined
Nov 13, 2006
Messages
15,436 (2.43/day)
Location
Mid-Atlantic
System Name Desktop
Processor i5 13600KF
Motherboard AsRock B760M Steel Legend Wifi
Cooling Noctua NH-U9S
Memory 4x 16 Gb Gskill S5 DDR5 @6000
Video Card(s) Gigabyte Gaming OC 6750 XT 12GB
Storage WD_BLACK 4TB SN850x
Display(s) Gigabye M32U
Case Corsair Carbide 400C
Audio Device(s) On Board
Power Supply EVGA Supernova 650 P2
Mouse MX Master 3s
Keyboard Logitech G915 Wireless Clicky
Software The Matrix
It was only a matter of time before we saw this. I wonder how this will effect IBM and Intel's sales.
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
26,956 (3.71/day)
Processor Ryzen 7 5700X
Memory 48 GB
Video Card(s) RTX 4080
Storage 2x HDD RAID 1, 3x M.2 NVMe
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 10 64-bit
It was only a matter of time before we saw this. I wonder how this will effect IBM and Intel's sales.

if people stop using the cloud [for hosting], many of them will use dedicated servers, which means installed server count goes up
 

johnspack

Here For Good!
Joined
Oct 6, 2007
Messages
5,979 (0.99/day)
Location
Nelson B.C. Canada
System Name System2 Blacknet , System1 Blacknet2
Processor System2 Threadripper 1920x, System1 2699 v3
Motherboard System2 Asrock Fatality x399 Professional Gaming, System1 Asus X99-A
Cooling System2 Noctua NH-U14 TR4-SP3 Dual 140mm fans, System1 AIO
Memory System2 64GBS DDR4 3000, System1 32gbs DDR4 2400
Video Card(s) System2 GTX 980Ti System1 GTX 970
Storage System2 4x SSDs + NVme= 2.250TB 2xStorage Drives=8TB System1 3x SSDs=2TB
Display(s) 2x 24" 1080 displays
Case System2 Some Nzxt case with soundproofing...
Audio Device(s) Asus Xonar U7 MKII
Power Supply System2 EVGA 750 Watt, System1 XFX XTR 750 Watt
Mouse Logitech G900 Chaos Spectrum
Keyboard Ducky
Software Manjaro, Windows 10, Kubuntu 23.10
Benchmark Scores It's linux baby!
Well, the ROF master game server is back up thank god, and so is the store. 777 Studios was having a big sale as Easter weekend is their biggest selling time of the year. They lost a load of money on this. They count on the revenue to continue building up the game engine, and we are waiting for some really cool new features expected in the next patch. This really screwed a lot of people....
Edit: our main forum is still down however.
 

Kreij

Senior Monkey Moderator
Joined
Feb 6, 2007
Messages
13,817 (2.21/day)
Location
Cheeseland (Wisconsin, USA)
if people stop using the cloud [for hosting], many of them will use dedicated servers, which means installed server count goes up

True .. unless the hosting companies are renting space in the cloud for the data so they do not have to manage their own infrastructure.
Best to ask if your hosting company is using their own servers for the hosting services.
 
Joined
Nov 25, 2008
Messages
926 (0.17/day)
Location
Akron, OH
System Name Main Rig
Processor Athlon 5350
Motherboard AsRock mITX
Memory 4gb
Storage 120gb Kingston HyperX SSD
Display(s) Samsung Syncmaster 740N
Power Supply Corsair 430 watt
I understand the frustration, but I think the frustration or fear of cloud computing is totally unmerited. You're an idiot if you think any system ever devised by man is infallible. You have to allow for downtime, both planned and unplanned, whether it be for cloud computing or for that shiny new Camaro you just bought--there is going to come a time when it just won't work when you need it to.

At the company where I work, we rely on LogMeIn to remotely control and diagnose the machines in our customer base. This past Sunday, they made some kind of backend change to their product that completely killed off our ability to control about 1800 machines remotely. It's not evil or the fault of the "cloud". It just happens.
 
Last edited:

Frick

Fishfaced Nincompoop
Joined
Feb 27, 2006
Messages
18,914 (2.86/day)
Location
Piteå
System Name Black MC in Tokyo
Processor Ryzen 5 5600
Motherboard Asrock B450M-HDV
Cooling Be Quiet! Pure Rock 2
Memory 2 x 16GB Kingston Fury 3400mhz
Video Card(s) XFX 6950XT Speedster MERC 319
Storage Kingston A400 240GB | WD Black SN750 2TB |WD Blue 1TB x 2 | Toshiba P300 2TB | Seagate Expansion 8TB
Display(s) Samsung U32J590U 4K + BenQ GL2450HT 1080p
Case Fractal Design Define R4
Audio Device(s) Line6 UX1 + some headphones, Nektar SE61 keyboard
Power Supply Corsair RM850x v3
Mouse Logitech G602
Keyboard Cherry MX Board 1.0 TKL Brown
VR HMD Acer Mixed Reality Headset
Software Windows 10 Pro
Benchmark Scores Rimworld 4K ready!
Meh, this stuff can happen without the help of cloud computing anyway, so I don't think it's the fault of the Cloud. I like the cload as a concept.
 

Easy Rhino

Linux Advocate
Staff member
Joined
Nov 13, 2006
Messages
15,436 (2.43/day)
Location
Mid-Atlantic
System Name Desktop
Processor i5 13600KF
Motherboard AsRock B760M Steel Legend Wifi
Cooling Noctua NH-U9S
Memory 4x 16 Gb Gskill S5 DDR5 @6000
Video Card(s) Gigabyte Gaming OC 6750 XT 12GB
Storage WD_BLACK 4TB SN850x
Display(s) Gigabye M32U
Case Corsair Carbide 400C
Audio Device(s) On Board
Power Supply EVGA Supernova 650 P2
Mouse MX Master 3s
Keyboard Logitech G915 Wireless Clicky
Software The Matrix
The cloud concept is great for saving power and money, but as we just saw, if the system goes down IT ALL GOES DOWN. so where amazon was offering money savings through some redundancy they may have to add even more now negating any big saving.
 

Kreij

Senior Monkey Moderator
Joined
Feb 6, 2007
Messages
13,817 (2.21/day)
Location
Cheeseland (Wisconsin, USA)
It's not evil or the fault of the "cloud". It just happens.

You're right. Unplanned outages can occur.
I'll remember to tell my customers that due to a unforseen consequence and the fact that I failed to set up an adequate contigency plan to even give them partial service in the event of a catastrophic failure, that, "Hey ... shit happens."

I'm sure they'll be understanding and quite forgiving.
 

johnspack

Here For Good!
Joined
Oct 6, 2007
Messages
5,979 (0.99/day)
Location
Nelson B.C. Canada
System Name System2 Blacknet , System1 Blacknet2
Processor System2 Threadripper 1920x, System1 2699 v3
Motherboard System2 Asrock Fatality x399 Professional Gaming, System1 Asus X99-A
Cooling System2 Noctua NH-U14 TR4-SP3 Dual 140mm fans, System1 AIO
Memory System2 64GBS DDR4 3000, System1 32gbs DDR4 2400
Video Card(s) System2 GTX 980Ti System1 GTX 970
Storage System2 4x SSDs + NVme= 2.250TB 2xStorage Drives=8TB System1 3x SSDs=2TB
Display(s) 2x 24" 1080 displays
Case System2 Some Nzxt case with soundproofing...
Audio Device(s) Asus Xonar U7 MKII
Power Supply System2 EVGA 750 Watt, System1 XFX XTR 750 Watt
Mouse Logitech G900 Chaos Spectrum
Keyboard Ducky
Software Manjaro, Windows 10, Kubuntu 23.10
Benchmark Scores It's linux baby!
Heh, yeah I use logmein as well to remotely control a server on the us east coast. It's a paid job, not good pay, but it's work. If logmein went down, I've have a very serious problem....
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
From the mouth of the lion: Amazon S3 Availability Event: July 20, 2008

And the single point of failure that brought the whole thing down was corrupt bits in non-errorchecking environment (it is now). The servers were giving incorrect information to each other so the other servers were trying to compensate for it. Eventually, the whole network of systems was bogged down by bogus overhead.

It was a programming error.


The cloud concept is great for saving power and money, but as we just saw, if the system goes down IT ALL GOES DOWN. so where amazon was offering money savings through some redundancy they may have to add even more now negating any big saving.
It doesn't all go down 99.7% of the time. That only happens when there is a failure in inter-server communication which is quite rare.
 
Last edited:

Easy Rhino

Linux Advocate
Staff member
Joined
Nov 13, 2006
Messages
15,436 (2.43/day)
Location
Mid-Atlantic
System Name Desktop
Processor i5 13600KF
Motherboard AsRock B760M Steel Legend Wifi
Cooling Noctua NH-U9S
Memory 4x 16 Gb Gskill S5 DDR5 @6000
Video Card(s) Gigabyte Gaming OC 6750 XT 12GB
Storage WD_BLACK 4TB SN850x
Display(s) Gigabye M32U
Case Corsair Carbide 400C
Audio Device(s) On Board
Power Supply EVGA Supernova 650 P2
Mouse MX Master 3s
Keyboard Logitech G915 Wireless Clicky
Software The Matrix
I understand how servers work. This crash was completely unacceptable. If you build up an entire server cloud system around "reliability, efficiency and flexibility" then it should never ever go down like that. There should have been checks in place or some kind of policy that requires the admins to check for irregular activity. That could not have possibly gone unnoticed for long but who knows...
 

FordGT90Concept

"I go fast!1!11!1!"
Joined
Oct 13, 2008
Messages
26,259 (4.65/day)
Location
IA, USA
System Name BY-2021
Processor AMD Ryzen 7 5800X (65w eco profile)
Motherboard MSI B550 Gaming Plus
Cooling Scythe Mugen (rev 5)
Memory 2 x Kingston HyperX DDR4-3200 32 GiB
Video Card(s) AMD Radeon RX 7900 XT
Storage Samsung 980 Pro, Seagate Exos X20 TB 7200 RPM
Display(s) Nixeus NX-EDG274K (3840x2160@144 DP) + Samsung SyncMaster 906BW (1440x900@60 HDMI-DVI)
Case Coolermaster HAF 932 w/ USB 3.0 5.25" bay + USB 3.2 (A+C) 3.5" bay
Audio Device(s) Realtek ALC1150, Micca OriGen+
Power Supply Enermax Platimax 850w
Mouse Nixeus REVEL-X
Keyboard Tesoro Excalibur
Software Windows 10 Home 64-bit
Benchmark Scores Faster than the tortoise; slower than the hare.
It was like a virus. Once the problem took hold, it spread. The only way to stop it was to reboot the entire cloud. It shares a lot in common with that Northeast Corridor power failure in 2000 (I think). One transformer/database went down and the rest responded as they were programmed to but, instead of isolating the issue in both cases, they magnified it.

You can't blame them for being human.
 

Easy Rhino

Linux Advocate
Staff member
Joined
Nov 13, 2006
Messages
15,436 (2.43/day)
Location
Mid-Atlantic
System Name Desktop
Processor i5 13600KF
Motherboard AsRock B760M Steel Legend Wifi
Cooling Noctua NH-U9S
Memory 4x 16 Gb Gskill S5 DDR5 @6000
Video Card(s) Gigabyte Gaming OC 6750 XT 12GB
Storage WD_BLACK 4TB SN850x
Display(s) Gigabye M32U
Case Corsair Carbide 400C
Audio Device(s) On Board
Power Supply EVGA Supernova 650 P2
Mouse MX Master 3s
Keyboard Logitech G915 Wireless Clicky
Software The Matrix
oh i dont blame anyone. and i like cloud technology. but if amazon thinks that their cloud business will one day bring in more revenue for them than sales on amazon.com (which is what their ceo says they want) then they better start adding redundancy and people better get used to paying a little bit more for it.
 
Top