GTX 1070 Firmware Overwritten by Malware - Unable to Reset

[Folterknecht]

Please keep us updated how this plays out!

 

[delshay]

If the mb bios is infected with something just trying to reflash it like you normaly update a bios can easily do nothing. The moment when you boot from the usb stick is too late (you risk to have the usb stick infected also). Basicaly you need to remove the bios chip and flash it with an SPI writer or just replace it with an identical chip that has the right bios flashed. Awfull part here is that now the BIOS/UEFI chips are soldered on the motherboard.

Based on the stuff I read here the gpu bios looks to be clear. (Assuming here that the bios reader program is not tricked to read something else and not the actual bios wrote on the gpu bios chip...)

The ram spd is just too small to actually have something there even if you remove everything not needed and keep the bare minimum on it. Yes this can be flashed also, I've done it years ago with an SDRAM module that had a real mess there. As far as I know the spd is only read to set the stuff regarding the ram in bios and nothing else.

Then we have the hdd/ssd firmware that can be altered to do nasty things. You need the chip with the firmware removed from the hdd/ssd and flashed with an SPI writer and this is way harder to do compared to the BIOS/UEFI chip.

Also each of them need to be reflashed without being connected to any of the potential infected hardware. Cleaning something like this will be really painfull. As others have already said such type of attack needs some real motivation, also the one behind doesn't look to actually care if it's gonna be detected (this remove some particular actors that well won't want to be detected to begin with).

In theory any firmware that is stored on chip, flash or whatever big enough can be altered to do nasty things (the big enough is the key).

That basically what I was trying to say about SPD. If you had a 2GB memory module & a virus change the size to 4GB memory module, system will not boot. What I was asking, can a virus do this? all it has to do is change a single Byte.

I know nothing about viruses & I have never been affected because I don't open strange Emails or go to funny site. If I do bounce into a strange website my virus checker blocks it automatically. There even a warning before I even click on dodge website.

 

[TheoneandonlyMrK]

I mean, I can't say much due to slander concerns (need concrete evidence), but I will say we have a good idea of who did this, their motivations, and it's signifigantly less exciting than you guys think.

The tech is fascinating, sure, but most of it has been downloadable in at least proof of concept form for years. This isn't a government level actor IMO, and even if it was, sorry, but I won't stop until I get a legal order (or convincing request) telling me to do so. He came asking for help and I believe in "innocent until proven guilty."



I really doubt spectre was utilized here. The best advice I can give for avoiding this is not letting malware get to your rig. It all starts as a userland admin level flash command...

Very interesting read but very bad tale, sorry for the Op's miss fortune here, i've had to deal with many peoples infected pc but if it were ever at this level id be using a hammer since i have to admit i would doubt i had any other way of stopping it spreading, once its within uefi i'd try,, sure, but your efforts are beyond my skills, hopefully this is an isolated thing, I would hate to see it in life, Good luck to you and the Op.

 

[Deleted member 163934]

That basically what I was trying to say about SPD. If you had a 2GB memory module & a virus change the size to 4GB memory module, system will not boot. What I was asking, can a virus do this? all it has to do is change a single Byte.

It's been 10+ years since I've wrote the SPD of a memory module. I do remember I messed up a bit with the content of it until I figure out what I supposed to change and things didn't really broke when I wrote bad things on the SPD, but maybe the motherboard just didn't cared that much about the content of the SPD (it was an old 440BX motherboard, I still have it in the house but no cpu and no ram for it). I have a motherboard that just refuses to boot if I touch any of the settings of the current DDR3 installed (it runs that 1333 Mhz DDR3 @ 800Mhz), as soon as I touch any setting (even increasing the timings) regarding memory it just no longer want to boot and the ram is good cause it happy works at the right freq without problems in another pc and the motherboard is also good cause if I install other DDR3 it works properly and allow me to play with the memory settings... (I suspect that the motherboard has issues with the RFC value in the SPD of the ram, it's 200 for 1333 Mhz, the highest from all the DDR3 I have and the bios doesn't have 1 single setting for RFC I have RFC1 and RFC2 that are not identical and still I can't explain why even changing CL from 6 at 9 and nothing else regarding RAM makes it refuse to boot... I suspect the RFC value because the rest is identical with the ram that has no issue...)

L.E.: I might be wrong but I think I've done with with a program with the ram installed in the pc. I can easily be wrong because I don't really remember much.

L.E. 2: This is not what I used (it was a dos exe, wasn't even working in windows, I remember I could only dump the content of the SPD and write it from a file, it was a pain to figure out how to fix the stuff in the spd and the checksum ). Anyway the ideea is that you can change the stuff there.
https://www.techpowerup.com/forums/threads/spdtool-read-edit-and-flash-your-memorys-spd.20349/

If you can change the timings then yes you can make the pc not boot by setting some timings that will make that ram give insta errors.

Joking: The only way I can be sure my father is not messing up a PC is by installing Linux on his pc. If he want to change something that he shouldn't he will need the password and I'm not telling him the password.

 

[delshay]

It's been 10+ years since I've wrote the SPD of a memory module. I do remember I messed up a bit with the content of it until I figure out what I supposed to change and things didn't really broke when I wrote bad things on the SPD, but maybe the motherboard just didn't cared that much about the content of the SPD (it was an old 440BX motherboard, I still have it in the house but no cpu and no ram for it). I have a motherboard that just refuses to boot if I touch any of the settings of the current DDR3 installed (it runs that 1333 Mhz DDR3 @ 800Mhz), as soon as I touch any setting (even increasing the timings) regarding memory it just no longer want to boot and the ram is good cause it happy works at the right freq without problems in another pc and the motherboard is also good cause if I install other DDR3 it works properly and allow me to play with the memory settings... (I suspect that the motherboard has issues with the RFC value in the SPD of the ram, it's 200 for 1333 Mhz, the highest from all the DDR3 I have and the bios doesn't have 1 single setting for RFC I have RFC1 and RFC2 that are not identical and still I can't explain why even changing CL from 6 at 9 and nothing else regarding RAM makes it refuse to boot... I suspect the RFC value because the rest is identical with the ram that has no issue...)

L.E.: I might be wrong but I think I've done with with a program with the ram installed in the pc. I can easily be wrong because I don't really remember much.

L.E. 2: This is not what I used (it was a dos exe, wasn't even working in windows, I remember I could only dump the content of the SPD and write it from a file, it was a pain to figure out how to fix the stuff in the spd and the checksum ). Anyway the ideea is that you can change the stuff there.
https://www.techpowerup.com/forums/threads/spdtool-read-edit-and-flash-your-memorys-spd.20349/

If you can change the timings then yes you can make the pc not boot by setting some timings that will make that ram give insta errors.

Most timings can be overridden by the BIOS & it will still work. In your link if a virus changed where it says "number of row or column address" the memory module will stop working. It just needs to change a single digit (bit). I think the same applies also to the Module Data Width. There are just some parts of the SPD that you cannot change, but it does not answer my question. Is it possible for a virus to target a specific part of the SPD & render the Memory Module inoperable.

 

[Deleted member 163934]

Is it possible for a virus to target what I have just pointed out.

As long as there is a tool that can do it from Windows there is a way to make a virus do it also.
Every bios/firmware of a component that is in your pc that can be updated from windows/dos/linux can also be wrote by a virus. Keep in mind that I used can because that's in theory. In practice there will be a lot of problems to solve and it will probably be a personalized attack (to make a generic attack the stuff used will have to be really big).
The chances for some random guy from the internet that doesn't really has anything interesting on his pc to have to deal with such thing are kinda low, unless someone really goes crazy and just want to destroy for no reason at all. At least in my personal case it will kinda be waste of resources to make a personalized attack because: I don't have nothing interesting on my pc, he can steal some game accounts (some of them will become inactive in a couple of months anyway so it's clear how much I care), I don't pay online using a credit/debit card (I prefer paysafe, if something goes bad the loss is limited) and the hardware I own is not that good for mining (heck I'd notice it when I start a game ).

If it reports other size compared to the real size of the ram... If it reports less compared to the real amount of it you have some chances for it to work (I had a 256 MB SDRAM single sided module installed in a mb that needed double sided modules to actually see 256 SDRAM (chipset limitations) and that module was seen as 128 MB SDRAM and it was working without any problems; yes I know really old information but that's the last time when I tried such things, I learned the lesson that way; overwriting the ram amount in Linux was actually working, that's the only way I found out to actually see and use the entire ram, don't ask me how or why I have no clue (it was only the mb chipset that was failing to initialize it properly, probably linux was overwriting the entire memory table and the cpu was happy using the new one or something like that)), if it reports more it will probably not work. It probably depends from motherboard to motherboard, already said I have a working motherboard and a pair of working DDR3 that work together only with everything on auto and that's @ 800 Mhz not at the rated 1333 Mhz and if I touch any setting in bios regarding RAM (heck changing only CL from 6 (like it is on auto) to 9 should do no harm) it just no longer wants to boot (I even put all the timings to max keeping the freq @ 800 mhz and it still fails to boot, don't ask me why because the manual set timings are considerable higher (relaxed) compared to what it sets on auto...) and to get it to boot again I need to do a jumper bios reset...

If the numbers of ranks are wrote wrong but the size of the memory is right it should work without problems (like it say 1 rank 8 banks when it's actuallty 2 ranks 8 banks), I have a pair that came like this and it has no issue (on 3 distinct mb).

If the stuff needed for the initialization of the RAM is wrong in the SPD then it's about luck mostly depending how permissive or not is the MB bios/uefi. What will work in a mb can easily make other mb (not same model/bios version) not boot...
For sure if it's told to initialize more RAM then there is physicaly there there will be problem. When? For sure when the OS will try to use more than the physical amount (to be read write in an area that doesn't physicaly exist) but it can happen sooner. It should fail the bios/uefi memory test to be honest before even getting to OS.

I don't plan to do such tests . If I was rich and didn't cared if I break the hardware sure but that's not the case for me.

 

[MrGenius]

The level of speculation in this thread is beyond sensible. Nobody really knows what's going on here. And until someone with some actual expertise in malware gets involved, we never will. All due respect to R-T-B. But he doesn't actually know what he's dealing with. What he knows is he's compared samples of suspected "unhealthy" code to what are known to be "healthy" samples of said code, and found that they're different. But it's pretty easy for anyone to see where written information differs. In fact any one of us could draw the same conclusions he has. The relevant questions are: Different how? Different why? What's REALLY going on here? Is this an example of legitimate malware? Does it work? What is it designed to do if it does? We know nothing at this point. It could be perfectly harmless(as in non-functional, written by someone who doesn't know exactly what they're doing). It could be some asshole's sick idea of a joke(as in written by someone who does know exactly what they're doing, but is doing it just to fuck with this guy's head). It could be a lot of things. Of course, it could be your worst nightmare too. But is it? Really? What if it turns out that it's not?

 

[R-T-B]

The level of speculation in this thread is beyond sensible. Nobody really knows what's going on here. And until someone with some actual expertise in malware gets involved, we never will. All due respect to R-T-B. But he doesn't actually know what he's dealing with. What he knows is he's compared samples of suspected "unhealthy" code to what are known to be "healthy" samples of said code, and found that they're different. But it's pretty easy for anyone to see where written information differs. In fact any one of us could draw the same conclusions he has. The relevant questions are: Different how? Different why? What's REALLY going on here? Is this an example of legitimate malware? Does it work? What is it designed to do if it does? We know nothing at this point. It could be perfectly harmless(as in non-functional, written by someone who doesn't know exactly what they're doing). It could be some asshole's sick idea of a joke(as in written by someone who does know exactly what they're doing, but is doing it just to fuck with this guy's head). It could be a lot of things. Of course, it could be your worst nightmare too. But is it? Really? What if it turns out that it's not?

The answer to your question can’t be publically posted due to concerns of the malware being reused... but suffice to say I have seen a lot more than just “comparisons.” The suspected infected sections have at least a few strongly suspected sections due to incriminating strings present in the malware. ATM I am still unable to definitively prove it links to the userland malware, but I am working on that and getting there. When I do, I will contact the media and manufacturers to prove my claims privately.

I’m not able to say more now, but the management engine is indeed only a suspicion based on behavior for what it is worth. It is too obfuscated atm to determine more.

On that note, to avoid hype, I am going to stop the updates until I have proof. Hype and speculation serve no one.

 

[lexluthermiester]

There is even worse, what i don't understand is why the target is a single person

The OP might not be the intended target..

I kind of gleaned why someone would want to target him in discussions, I won't say more than that. I will say it's a legit job he works, and not something sketchy or weird, but lucrative to infect.

.. an idea supported by this.

BTW...this is why we use secure boot people. Well...in theory.

SecureBoot can be circumvented and is more hassle than it's worth sometimes.

Also if it was targeted and this in depth, this is FBI territory.

The FBI are in the minority of seriously "freaky-neaky" stuff like this. Seems more blackhat to me.

RTB I cant help but think you are making a mistake helping this guy. Call it the Miami in me, but helping strangers with really odd problems never works out well. Ever.

What are you talking about? I do it all the time. It's called tech support. I do it professionally for fee. @R-T-B is doing it for the challenge, for fun and to help someone out. The only other difference is that he's doing it remotely where as I do it in my shop when people bring their systems in.

Honestly I would hand this over to the FBI. If its legit they will handle it.

YOU need to go to the FBI. Not him. First man to the boat makes the deals. This could easily get tossed back on to you.

Now you're just being paranoid. The FBI won't care about something like this.

Helping strangers with odd problems is a big "no, no". Good way to end up in jail or dead.

Way paranoid. Seriously..

Keep in mind mailman is known around these parts for his.... can I just say "different bedside manner?"

That's putting it mildly..

The amount of dead sweet ssd you went through alone makes me want to cry...

Right?

But bottom line: I think short of hardware killing the management engine, we'll never defeat this thing. So yeah, he needs to send it in.

This whole thing is outside my normal area of expertise, but I have had some dealings with these kinds of exploits. This seems like some next-level sh!t going on here.

If intel were smart, they would keep ime as a separate chip, not built into the cpu, and that ime chip is behind a jumper on the motherboard.

Agreed. But the smart thing to do isn't always what is profitable..

this reeks of government malware

No it doesn't. Trust me on the one.

 

[MrGenius]

Speculation is fine. Within reason. And expecting the worst, while hoping for the best, is usually a better way to deal with reality. Just don't let those expectations, or those hopes, be all consuming. When conclusive results of an investigation(or evidential facts) haven't pointed directly to either end of the spectrum. That's all I'm trying to say.

@lexluthermiester Precisely. Hence the "in theory" and the .

 

[R-T-B]

When conclusive results of an investigation(or evidential facts) haven't pointed directly to either end of the spectrum. That's all I'm trying to say.

Good point. There's a lot of circumstantial evidence here, but I won't post again until I am certain.

Thanks for reading anyways, everyone. I'll let you know (when I actually "know" that is).

 

[MrGenius]

@R-T-B what I'm saying isn't even directed mainly at you. I feel like you're being very level-headed about the situation actually. It's everyone else jumping straight overboard with all the grand conspiracy theories and definitive statements about it being unquestionably some 31337 H4X0R/government sponsored/super-spy level targeted malware attack. Which it very well may be. Or very well may not be. I'm just trying to stress the "may not be" to balance things out here.

 

[Regeneration]

There is no conspiracy here. Government agencies are very paranoid of being exposed and very protective of their investment (private exploits). If the subject suspects anything, they'll take it off, remove any trace and switch to another method, or wait for things to cooldown. The exploits used here are public.

Not all viruses are lame adware or ransom crap. The ultimate ones are undetected and durable enough to survive OS reinstall. The best hackers are those who keep a low profile and fly under the radar. You don't see malware like this often because most "malware distributors" are script kiddies. The skilled hackers make good money working for data security firms (and plus they don't have to worry about cops knocking on their door in the middle of the night).

One day I was setting up a demo of "infected system"... took me hours to find proper malware.

 

[trparky]

The morbid curiosity in me has got me wondering just who the heck he pissed off. As another person in this thread said, one doesn't just get targeted with stuff like this unless someone has a really nasty grudge. I'll be waiting for a post-analysis report on this @R-T-B .

As his modem was the entry point by all appearances (an old outdated comcast modem with firmware loopholes started this I think) he should be extra vigilant there IMO

Is there a possibility that it could have gone up the chain all the way to the cable company's CMTS? Good God, if it did the cable company is going to have a mess on their hands.

 

[InVasMani]

Trojan horse attacks.

The morbid curiosity in me has got me wondering just who the heck he pissed off. As another person in this thread said, one doesn't just get targeted with stuff like this unless someone has a really nasty grudge. I'll be waiting for a post-analysis report on this @R-T-B .


Is there a possibility that it could have gone up the chain all the way to the cable company's CMTS? Good God, if it did the cable company is going to have a mess on their hands.

In a sense I think Comcast already does have a BIG problem. It's kind of their negligence on replacing these vulnerable outdated cable modems that was the initial attack vector. The cable companies should be expected to protect customers information and hardware with the services they are providing and if they are known to be flawed fixed or better yet replaced. I'm not sure fixed is enough what prevents them from flashing hardware firmware back to a vulnerable state w/o the user realizing it? It's even more concerning given the scope and reach of meltdown/spectre couldn't they simply revert those leaky patches back to a bad state again provided they had another attack vector to do so again? It only takes one bad judgment lapse to become the victim.

 

[qubit]

One day I was setting up a demo of "infected system"... took me hours to find proper malware.

Surprsing you couldn't find anything quickly. This reminds me of an experiment I ran around 2002/3.

This was in the days of USB ADSL modems - remember them? and 1Mb/s internet here in England. So, I took a spare PC and installed Windows XP on it. No service packs, no patches, just the original Swiss cheese version off the CD and didn't allow it to update. I then installed the network monitoring utility NetMeter on it and disabled the Windows firewall. I then installed the USB modem drivers and put it online, facing the raw, unfiltered internet. I did nothing else, no using a web browser, nothing.

Within a few minutes I started seeing weird popups on the desktop advertising dodgy things (I forget the details now it's so long ago). I left it like that for about 8 hours, after which NetMeter saw constant outbound traffic for no apparent reason, maybe 100Kb/s or so. Certainly Task Manager didn't show anything.

Windows still worked and I could browse the web, but it didn't feel quite right and those popups kept, well, popping up every so often. This was a visceral example of how easy it is to get infected if the usual precautions aren't taken. The very first precaution of course, is to have a hardware firewall between you and the internet, such as IPCop.

Experiment over, I then formatted the HDD and reconnected the USB modem to the network via my IPCop firewall. The funny thing is that I'm still on ADSL (fibre unavailable, boohoo) and I still have all the old hardware and software, so I could actually repeat the experiment now if I wanted to. The main difference is that USB modem would be working at its full 8Mb/s of course, now.

@dorsetknob @eidairaman1 @jboydgolfer @rtwjunkie @CAPSLOCKSTUCK @infrared @Norton I think you'll like this story.

 

[TheMailMan78]

The OP might not be the intended target..

.. an idea supported by this.

SecureBoot can be circumvented and is more hassle than it's worth sometimes.

The FBI are in the minority of seriously "freaky-neaky" stuff like this. Seems more blackhat to me.

What are you talking about? I do it all the time. It's called tech support. I do it professionally for fee. @R-T-B is doing it for the challenge, for fun and to help someone out. The only other difference is that he's doing it remotely where as I do it in my shop when people bring their systems in.


Now you're just being paranoid. The FBI won't care about something like this.

Way paranoid. Seriously..

That's putting it mildly..

Right?

This whole thing is outside my normal area of expertise, but I have had some dealings with these kinds of exploits. This seems like some next-level sh!t going on here.

Agreed. But the smart thing to do isn't always what is profitable..

No it doesn't. Trust me on the one.

The FBI won't care about this?

https://www.fbi.gov/investigate/cyber/national-cyber-investigative-joint-task-force

Apparently you're not as up to date as you think.

The fact @OneMoar and I agree on something should be enough to worry RTB.

 

[R-T-B]

My opinion is the FBI will care if the evidence is in easy-to-digest spoon-fed to them in a form that is irrefutable, and doesn't waste much time. That's the only reason I have not contacted them yet. I really need hard proof or I feel I will get ignored. Best foot forward, always.

I really don't want to continue this discussion right now, but I can't really abide conspiracy theories. I will repeat myself: I feel the motive to this is much more boring than any of you can possibly imagine.

 

[Space Lynx]

So a way to avoid all of this would be to not flash your GPU bios? neat, and noted. never saw the need to flash bios for 3% performance gain myself, not worth voiding warranty.

 

[R-T-B]

So a way to avoid all of this would be to not flash your GPU bios? neat, and noted. never saw the need to flash bios for 3% performance gain myself, not worth voiding warranty.

I still have no definitive proof of the origin, but the gpu being flashed is not even on the list, honestly.

 

[MrGenius]

So a way to avoid all of this would be to not flash your GPU bios? neat, and noted.

Huh? Nobody said anything about this being the result of flashing a GPU BIOS. Unless you know something we don't...

never saw the need to flash bios for 3% performance gain myself, not worth voiding warranty.

1. You can get A HELL OF A LOT MORE than 3% performance gain. It's called "uncorking the BIOS". And professional overclockers do it for a reason.

2. Manufacturers release BIOS updates for graphics cards too. And if you were to flash it back to the original BIOS before seeking a warranty claim, they'd have no real way of knowing. Spare me the debate about the ethics involved. I'm well aware it's not the most honest thing to do.

EDIT: There's a way they could know. But I highly doubt they keep a detailed record of the original BIOS checksums for every single card they sell, just so they can try and void your warranty if they don't match up.

 

[OneMoar]

you can't flash a pascal bios
moot point misleading thread title is misleading
its his motherboards bios that was compromised, I stand by the point there is something the op isn't telling us one does not simple get this level of malware out of the blue
a trip to china with the pc going though customs would be a example

 

[R-T-B]

point there is something the op isn't telling us

I think we've both confirmed that several times.

you can't flash a pascal bios

You actually can (My Titan XP is hardware flashed) but that is probably not the case here.

 

[eidairaman1]

Surprsing you couldn't find anything quickly. This reminds me of an experiment I ran around 2002/3.

This was in the days of USB ADSL modems - remember them? and 1Mb/s internet here in England. So, I took a spare PC and installed Windows XP on it. No service packs, no patches, just the original Swiss cheese version off the CD and didn't allow it to update. I then installed the network monitoring utility NetMeter on it and disabled the Windows firewall. I then installed the USB modem drivers and put it online, facing the raw, unfiltered internet. I did nothing else, no using a web browser, nothing.

Within a few minutes I started seeing weird popups on the desktop advertising dodgy things (I forget the details now it's so long ago). I left it like that for about 8 hours, after which NetMeter saw constant outbound traffic for no apparent reason, maybe 100Kb/s or so. Certainly Task Manager didn't show anything.

Windows still worked and I could browse the web, but it didn't feel quite right and those popups kept, well, popping up every so often. This was a visceral example of how easy it is to get infected if the usual precautions aren't taken. The very first precaution of course, is to have a hardware firewall between you and the internet, such as IPCop.

Experiment over, I then formatted the HDD and reconnected the USB modem to the network via my IPCop firewall. The funny thing is that I'm still on ADSL (fibre unavailable, boohoo) and I still have all the old hardware and software, so I could actually repeat the experiment now if I wanted to. The main difference is that USB modem would be working at its full 8Mb/s of course, now.

@dorsetknob @eidairaman1 @jboydgolfer @rtwjunkie @CAPSLOCKSTUCK @infrared @Norton I think you'll like this story.

There is ATM based ADSL and IP DSL, here it can be RT based or CO based, CO based is the worst due to attenuation.

 

[dorsetknob]

This was in the days of USB ADSL modems - remember them?

Still got my BT Supplied ADSL Speedtouch USB modem (working but retired as now have FTTC)