City of Baltimore goes down because of ransomware

[Solaris17]

The second time in a little over a year.

Davis said critical systems, including 911 and 311, were not been affected, but that the majority of city servers were shut down. The effects ranged from a City Council committee canceling a hearing on gun violence to water customers being unable to get billing questions answered.

Baltimore city government computer network hit by ransomware attack

Baltimore City government computers were infected with ransomware Tuesday, the mayor's office said, shutting down many technology systems while officials battle the attack.

www.baltimoresun.com

Cool facts, Most cities have municipal intranets that control core city systems, from cameras and stop lights to city broadcasts or emergency services. They are also historically tampered with or vulnerable.

 

[Batou1986]

We in the news for all sorts of good things lately ....

 

[Fangio1951]

This reminds me of - Die Hard 4 - where terrorists systematically shut down the US

 

[remixedcat]

We in the news for all sorts of good things lately ....

Is Rosedale still clean? or has it gone ghetto? Had fam there. I miss my uncle (died when I was 13 I think)...

Also I think they downgraded their routers or somethin. I heard they implmented some Fortinets and well.. I got a review for that and they have flaky firmware.

 

[Ferrum Master]

What is the ultimate motive? Just showing off?

 

[lexluthermiester]

What is the ultimate motive? Just showing off?

I'll bet someone had a grudge against the city of Baltimore.

 

[Bill_Bright]

What is the ultimate motive?

Typically the payoff.

It should be noted while this is the 2nd attack against Baltimore in just over a year, it is a different system that was hit. But is that good or bad? I don't know. It is really bad if the same vulnerability was used to bring both systems down.

 

[lexluthermiester]

So unfortunate in their area. I have some relatives in Vernon and was told that that Robinhood ransonware really hit hard Baltimore. Specially government websites. Damage costs around $18M. I hope they provide more cyber security solutions.

The fault of this rests with the city admins when they didn't listen to their IT admins when the advice was given to spend the time & money on proper security measures.

 

[Bill_Bright]

The fault of this rests with the city admins when they didn't listen to their IT admins when the advice was given to spend the time & money on proper security measures.

I sympathize with governments (big and small) and businesses (big and small) and just about every other organization on this. The problem with spending fortunes on IT and IT security is those are pure money pits. They are huge expenses that don't "make" money for the organization. They don't "break even". They just spend, spend, spend. And when that is tax payer money - which typically is in short supply already - it becomes even more painful.

Sadly, as is too often the case (Equifax comes to mind), much of the fault also lies with the IT admins too. Yes, in this latest Baltimore attack, city managers failed to fund the IT systems to upgrade to more modern systems. But it seems IT admin also failed to keep their current systems updated with available patches that could have prevented the infection in the first place - exactly the problem with Equifax.

As noted here,

The city's computer systems were outdated and IT experts failed to install adequate updates to protect against attacks, which would have stopped its spread.

 

[Solaris17]

The problem with spending fortunes on IT and IT security is those are pure money pits. They are huge expenses that don't "make" money for the organization. They don't "break even". They just spend, spend, spend. And when that is tax payer money - which typically is in short supply already - it becomes even more painful.

That’s not true, and frankly that’s the thinking of the management that caused this issue.

IT is a force multiplier.

How much money has Baltimore lost from this?
I am willing to bet it’s more than those perimeter firewalls and AV licenses that were denied.

How much productivity does it cost when Sarah in accounting is down and can’t do payroll because someone has to spend 2 hours driving out to the location and then format her machine with a thumb drive?

I bet it costs the company more to pay Sarah for doing nothing, pay the IT guy for driving and pay both to wait for the unit.

Has the company spent more in this scenario? No. Both people would have been on shift anyway. But the company paid for both Sarah and IT guy for not doing their respective jobs in those 3 hours.

Is that more than the server with deployment services installed that also got denied?

I’m going to say no to that too.

That thinking is out moded and incorrect. IT is no longer a cost center because we no longer use type writers at work.

It is a fallacy to starve these departments of funding. It comes down to this mode of thought combined with people that have no idea what IT encompasses that breed more people into thinking this and eventually to situations like we see here.

 

[Bill_Bright]

That’s not true, and frankly that’s the thinking of the management that caused this issue.

Of course its true. You are thinking about this backwards and not understanding what I said.

What revenue do the "IT Security" departments for The Home Depot, the City of Baltimore or St Mary's Hospital make for those organizations? None!

Preventing losses is not "making" money. It is spending money to keep from losing more money.

Of course it is management's fault. They failed to fund IT Security. And they failed to impress upon the IT departments to apply patches in a timely basis.

Spending money on IT security is absolutely essential. Money for current hardware, current software, personnel and training are absolutely essential. But what revenue does any of that bring in to the organization? Nada. Zilch. $0.00.

It is just like buying insurance. It is spending money with no money in return. It may save you money, but it does not make you money.

For a company, they must sell products or services to make money. What product or service does the IT Department of the City of Baltimore "sell" to make money? None! But taxpayers must pay for salaries, facilities, hardware and software and more.

It is a fallacy to starve these departments of funding.

Having spent almost 50 years as a technician in IS/IT departments supporting federal, state and local governments, as well as private companies, small businesses and individuals, I agree 100% with that. But that funding must come from somewhere and for sure, the IT department in the City of Baltimore is not generating it. Taxpayers are and city managers are the ones doling that funding out. And city managers (and IT management) are the ones setting policy for applying patches and updates.

 

[Solaris17]

but it does not make you money.

Sure it does, it makes you money when everyday IT tasks that generally cause any monute of downtime, are all of a sudden rendered obsolete because of advancements in infrastructure.

You might have a hard time with business fundamentals because spending money to save does generate money, because you are "buying" time.

For example if the normal operating procedure is baltimore formatting machines one at a time and we will say this takes 30min and then they spend an hour installing all the software the user needs we will be conservative and say this only takes an hour. Followed of course by the updates, we will be conservative again and say this only takes 30min. Baltimore SOP is to take 2 hours of downtime to setup a user machine (budgeted).

However, this is pay on both the employees (labor). By spending $ on even a whitebox server and setting up MDT to auto install software and WDS to deploy the image itself, you have brought that down to the 30min it only took to format the unit.

This means they have gotten back via a change in SOP 1:30min of labor that can go to the "next" project that otherwise would have been started tomorrow. Allowing the user to generate more money for the company.

Process generates revenue. Does IT walk door to door selling? No, but if they leverage tools to make employees more efficient by saving LABOR TIME then absolutely IT not only saves money, but helps generate revenue by giving them the only thinkg you cant buy, time.

 

[dirtyferret]

Sure it does, it makes you money when everyday IT tasks that generally cause any monute of downtime, are all of a sudden rendered obsolete because of advancements in infrastructure.

You might have a hard time with business fundamentals because spending money to save does generate money, because you are "buying" time.

For example if the normal operating procedure is baltimore formatting machines one at a time and we will say this takes 30min and then they spend an hour installing all the software the user needs we will be conservative and say this only takes an hour. Followed of course by the updates, we will be conservative again and say this only takes 30min. Baltimore SOP is to take 2 hours of downtime to setup a user machine (budgeted).

However, this is pay on both the employees (labor). By spending $ on even a whitebox server and setting up MDT to auto install software and WDS to deploy the image itself, you have brought that down to the 30min it only took to format the unit.

This means they have gotten back via a change in SOP 1:30min of labor that can go to the "next" project that otherwise would have been started tomorrow. Allowing the user to generate more money for the company.

Process generates revenue. Does IT walk door to door selling? No, but if they leverage tools to make employees more efficient by saving LABOR TIME then absolutely IT not only saves money, but helps generate revenue by giving them the only thinkg you cant buy, time.

I agree with all your points (and Bills) but there are few politicians who could survive re-election after spending money in municipality IT infrastructure when schools are under funded, teachers lack adequate pay, lack of police officers, opioid epidemic , hospitals are understaffed, clinics are needed, civil infrastructure needs repair., etc., etc.,

 

[Bill_Bright]

Sure it does, it makes you money when everyday IT tasks that generally cause any monute of downtime, are all of a sudden rendered obsolete because of advancements in infrastructure.

No! None of that is generating revenue.

You might have a hard time with business fundamentals because spending money to save does generate money, because you are "buying" time.

I'm afraid you don't understand business fundamentals. I have owned my own [successful] "IT Consulting, Custom PCs, and Computer Repair" business. There is a HUGE difference between the cost of doing business and making money. It takes money - lots of money to "invest" in that infrastructure (facilities, vehicles, LAN, computers, test equipment, work benches, tools and personnel and liability and health insurance). Only the people "make" money and that is only when they are busy selling a product or service for a profit. And what is profit? It is the money left over after all those expenses.

And that is quite different from running a government organization - which I have also done as the NCOIC (non-commissioned officer in charge) of several IS/IT units in my 24 year career in the USAF where everything we did cost taxpayers' money while generating $0.00 in return.

Once again, you equate saving money to making money. That is not how it works when running a business or organization. Saving money is just that. Saving money. Saving money is good! And certainly important. But not when spending some of it could have saved more in the long run. And that's where management failed here.

IF City of Baltimore management had spent more money on IT security and IF City of Baltimore and IT management had stressed the importance of applying patches on a timely basis, they would have "saved" the taxpayers the expense they are going through now.

Since this latest Baltimore ransomware event could have been prevented had IT applied the available patches (besides the bad guys) who do you say is at fault? I say city and IT management but say that is not true. So who is?

there are few politicians who could survive re-election after spending money in municipality IT infrastructure when schools are under funded, teachers lack adequate pay, lack of police officers, opium epidemic, hospitals are understaffed, clinics are needed, civil infastructure needs repair., etc., etc.,

This is very true too. But who's at fault here? Some fault, for sure goes to fraud, waste and abuse. And for sure, some to stupid pork barrel projects. But some must also go to the voters who refuse to vote for the necessary taxes, bonds, etc. to pay for those municipal necessities. I hate paying taxes, but I always vote for school bonds in my district. But sadly, they rarely pass.

 

[dorsetknob]

Spending $ on IT Security and associated infrastructure is as IMPORTANT as Spending on your Fire/Police Services
None of them Generate Revenue..................But what happens if you Fail to Fund them (PROPERLY).

No need to Reply as SENSIBLE PEOPLE KNOW THE ANSWER

Spoiler

Crime Rise's............. Things Burn Down............Systems get compromised

 

[lexluthermiester]

The key problem is finding a balance. Baltimore has learned the hard way what will happen if you neglect your technology infrastructure. They will be a very stark example for governments and business's alike to remember in future.

I agree with all your points (and Bills) but there are few politicians who could survive re-election after spending money in municipality IT infrastructure when schools are under funded, teachers lack adequate pay, lack of police officers, opioid epidemic , hospitals are understaffed, clinics are needed, civil infrastructure needs repair., etc., etc.,

Agreed. A lot of good points have been made. Life has evolved a bit and politicians are still playing catch-up in many ways.

 

[Bill_Bright]

Baltimore has learned the hard way what will happen if you neglect your technology infrastructure.

Let's hope so. Remember, this is their 2nd ransomware attack in 18 months. It seems they didn't learn after the first. Lets hope they do now. And for the record, while it will probably cost taxpayers more during this crisis/attack, for the recovery, and for whatever it takes to implement the necessary tools, training and hardware to prevent recurrence, I personally applaud them for not giving in and paying the ransom.

and politicians are still playing catch-up in many ways.

I wish/hope that were true. But it seems to me most politicians are playing only whatever game it takes to get re-elected. Not to do what's right for their constituents. But that's for another discussion.

 

[lexluthermiester]

Remember, this is their 2nd ransomware attack in 18 months. It seems they didn't learn after the first. Lets hope they do now.

I don't think the city leaders understood it.

But it seems to me most politicians are playing only whatever game it takes to get re-elected.

Some of them actually care. Again, they all have to play a very careful balancing act. When everyone is screaming for more money and there's only so much to go around what do you do? In Baltimore's case the solution was an inexpensive hardware/software reconfig.

 

[Bill_Bright]

I don't think the city leaders understood it.

I am sure it was explained very thoroughly and they either chose to ignore the warnings from the press and their IT department or just too dumb to understand. And if the later, then they should have just heeded the warnings of the experts. That's why they have them.

When everyone is screaming for more money and there's only so much to go around what do you do?

I agree, that's the pickle. That goes back to my comment above about voters needing to swallow the bitter pill and cough up more money in taxes if we want potholes fixed, streets patrolled with better trained cops, our children getting the best education, etc. etc.

 

[lexluthermiester]

or just too dumb to understand.

This..

And if the later, then they should have just heeded the warnings of the experts. That's why they have them.

Completely agree.