• We've upgraded our forums. Please post any issues/requests in this thread.

Email spammer on my server

Joined
Sep 1, 2007
Messages
316 (0.08/day)
Likes
27
Location
UK
System Name Moose 1055
Processor AMD Phenom II X6 1055T
Motherboard ASRock 990FX Extreme3
Cooling Thermaltake PW 850i Liquid Cooler and loads of Case Fans
Memory OCZ (2x2GB) + Crucial (2x4GB) = 12GB DDR3 1600MHz
Video Card(s) Powercolor R9 285
Storage Corsair BX100 250GB + 2 x Western Digital Caviar Black 750GB
Display(s) Hanns G LCD 22" + Acer LCD 22"
Case NZXT Hades Mid Tower Case
Audio Device(s) Realtek ALC892
Power Supply Antec Earthwatts 650W
Mouse Gigabyte M6900
Software Windows 10 Enterprise
#1
My server (Ubuntu 12.04) has recently been unable to send emails as it's IP has been blocked due to it being reported for email spam. I decided to investigate and was not pleased by what I discovered!

A wireshark capture revealed an email attempted to be sent about once every 10 seconds, further investigation seemed to show that sshd sessions were being initiated which were sending tons of emails, the sshd sessions also appeared to be connected to other ips who were presumably logged in?

The sshd sessions are called "sshd: root" so they are logged in as root, first thing I did was change the root password and remove all the keys.

Still sshd connections are being made and are sending emails! What can I do?
 

Athlon2K15

HyperVtX™
Joined
Sep 27, 2006
Messages
7,848 (1.92/day)
Likes
2,305
Location
O-H-I-O
Processor AMD Ryzen 7 1800x
Motherboard Asus Crosshair VI Hero
Cooling CH6 EK MonoBlock
Memory TridentZ 16GB DDR4 3600
Video Card(s) GTX 1080Ti EK Full Cover Block
Storage Samsung 960 Pro
Display(s) LG 34UC88 Curved Ultrawide
Case EVGA DG86
Power Supply Corsair RM850x
Mouse Asus Strix Evolve
Keyboard Asus Strix Claymore
#2
hit the power button? :)
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,056 (3.44/day)
Likes
17,958
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#3
look in the logs, check what's happening, fix it :)
 

Aquinus

Resident Wat-man
Joined
Jan 28, 2012
Messages
10,401 (4.85/day)
Likes
5,481
Location
Concord, NH
System Name Kratos
Processor Intel Core i7 3930k @ 4.2Ghz
Motherboard ASUS P9X79 Deluxe
Cooling Zalman CPNS9900MAX 130mm
Memory G.Skill DDR3-2133, 16gb (4x4gb) @ 9-11-10-28-108-1T 1.65v
Video Card(s) MSI AMD Radeon R9 390 GAMING 8GB @ PCI-E 3.0
Storage 2x120Gb SATA3 Corsair Force GT Raid-0, 4x1Tb RAID-5, 1x500GB
Display(s) 1x LG 27UD69P (4k), 2x Dell S2340M (1080p)
Case Antec 1200
Audio Device(s) Onboard Realtek® ALC898 8-Channel High Definition Audio
Power Supply Seasonic 1000-watt 80 PLUS Platinum
Mouse Logitech G602
Keyboard Rosewill RK-9100
Software Ubuntu 17.10
Benchmark Scores Benchmarks aren't everything.
#4
look in the logs, check what's happening, fix it :)
+1: But I would still kill sshd until he figures it out.

Still sshd connections are being made and are sending emails! What can I do?
A: disable sshd if you can work locally.
(sudo /etc/init.d/sshd stop)

B: Disable password authentication (biggest vulnerability in a *nix system IMHO.)
@ /etc/ssh/sshd_config
You want:
Code:
PasswordAuthentication no
RSAAuthentication yes
PubkeyAuthentication yes
#AuthorizedKeysFile     %h/.ssh/authorized_keys
PermitRootLogin no
C: Enable shared key auth (and only shared key auth,) and generate a public/private RSA key pair.
(ssh-keygen -b 4096)

D: Allowing SSH into root is also dangerous. I would disable root login in then sshd config.

E: Copy your public key somewhere and enable sshd and you should be all set. That way the only way a hacker can get in through SSH is if they have your private key.

One of the more common reasons that mail fails (not initially, but over time) is when DNS is not properly setup. Maybe you're missing or have a bad MX or PTR record and the email server keeps retrying. That will make mail servers reject your email very quickly after a little while.
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,056 (3.44/day)
Likes
17,958
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#5
I'm not even sure that SSH is the source of his problems
 

qubit

Overclocked quantum bit
Joined
Dec 6, 2007
Messages
14,551 (3.98/day)
Likes
8,058
Location
Quantum Well UK
System Name Quantumville™
Processor Intel Core i7-2700K at stock (hits 5 gees+ easily)
Motherboard Asus P8Z68-V PRO/GEN3
Cooling Noctua NH-D14
Memory 16GB (4 x 4GB Corsair Vengeance DDR3 PC3-12800 C9 1600MHz)
Video Card(s) Zotac GTX 1080 AMP! Extreme Edition
Storage Samsung 850 Pro 256GB | WD Green 4TB
Display(s) BenQ XL2720Z | Asus VG278HE (both 27", 144Hz, 3D Vision 2, 1080p)
Case Cooler Master HAF 922
Audio Device(s) Creative Sound Blaster X-Fi Fatal1ty PCIe
Power Supply Corsair HX 850W v1
Software Windows 10 Pro 64-bit
#6
I'd do a format and reinstall if it looks like the box has been rooted. Otherwise, fix the leak and monitor activity like a hawk.
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,056 (3.44/day)
Likes
17,958
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#7
I'd do a format and reinstall if it looks like the box has been rooted. Otherwise, fix the leak and monitor activity like a hawk.
+1, rooted = reinstall
 

Aquinus

Resident Wat-man
Joined
Jan 28, 2012
Messages
10,401 (4.85/day)
Likes
5,481
Location
Concord, NH
System Name Kratos
Processor Intel Core i7 3930k @ 4.2Ghz
Motherboard ASUS P9X79 Deluxe
Cooling Zalman CPNS9900MAX 130mm
Memory G.Skill DDR3-2133, 16gb (4x4gb) @ 9-11-10-28-108-1T 1.65v
Video Card(s) MSI AMD Radeon R9 390 GAMING 8GB @ PCI-E 3.0
Storage 2x120Gb SATA3 Corsair Force GT Raid-0, 4x1Tb RAID-5, 1x500GB
Display(s) 1x LG 27UD69P (4k), 2x Dell S2340M (1080p)
Case Antec 1200
Audio Device(s) Onboard Realtek® ALC898 8-Channel High Definition Audio
Power Supply Seasonic 1000-watt 80 PLUS Platinum
Mouse Logitech G602
Keyboard Rosewill RK-9100
Software Ubuntu 17.10
Benchmark Scores Benchmarks aren't everything.
#8
I'm not even sure that SSH is the source of his problems
I know a couple of people who have been compromised because SSH was open and it allowed password authentication. Always use a key-pair when ever possible and if you can, require it. I agree though, there could be a problem elsewhere but that doesn't mean you shouldn't fix a potential problem before it happens if it wasn't SSH.

I'd do a format and reinstall if it looks like the box has been rooted. Otherwise, fix the leak and monitor activity like a hawk.
+1, rooted = reinstall
This. Fixing the problem is only a stop-gap measure. If they're in root they can make it very easy to get back in short of you turning the machine off or taking it off the network. Take it off the network, back it up and nuke it. After you re-install though, make sure to not go too lenient on the security settings for things like SSH though. Don't need this happening again. Make sure to change your password that you use on this box as well, for all accounts that had sudo and the password for root.

Occasionally a connection from China will try to make its way into my network. You may want to consider blocking IP ranges that you know that should never contact your server.
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,056 (3.44/day)
Likes
17,958
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#9
Always use a key-pair when ever possible and if you can, require it
The fun starts when you lose your private key due to fuckup, HDD crash or similar. Also trojan on your system could steal the private key (just like a keylogger can steal your typed password).

SSH password logins are perfectly safe and probably 90% of unix systems run with it. Weak passwords are not.

We moved SSH to another port on our servers to get rid of random (chinese) people trying to bruteforce it.
 

Aquinus

Resident Wat-man
Joined
Jan 28, 2012
Messages
10,401 (4.85/day)
Likes
5,481
Location
Concord, NH
System Name Kratos
Processor Intel Core i7 3930k @ 4.2Ghz
Motherboard ASUS P9X79 Deluxe
Cooling Zalman CPNS9900MAX 130mm
Memory G.Skill DDR3-2133, 16gb (4x4gb) @ 9-11-10-28-108-1T 1.65v
Video Card(s) MSI AMD Radeon R9 390 GAMING 8GB @ PCI-E 3.0
Storage 2x120Gb SATA3 Corsair Force GT Raid-0, 4x1Tb RAID-5, 1x500GB
Display(s) 1x LG 27UD69P (4k), 2x Dell S2340M (1080p)
Case Antec 1200
Audio Device(s) Onboard Realtek® ALC898 8-Channel High Definition Audio
Power Supply Seasonic 1000-watt 80 PLUS Platinum
Mouse Logitech G602
Keyboard Rosewill RK-9100
Software Ubuntu 17.10
Benchmark Scores Benchmarks aren't everything.
#10
We moved SSH to another port on our servers to get rid of random (chinese) people trying to bruteforce it.
+1: Always a good choice. My personal favorite is 60031. :p
 
Joined
Sep 1, 2007
Messages
316 (0.08/day)
Likes
27
Location
UK
System Name Moose 1055
Processor AMD Phenom II X6 1055T
Motherboard ASRock 990FX Extreme3
Cooling Thermaltake PW 850i Liquid Cooler and loads of Case Fans
Memory OCZ (2x2GB) + Crucial (2x4GB) = 12GB DDR3 1600MHz
Video Card(s) Powercolor R9 285
Storage Corsair BX100 250GB + 2 x Western Digital Caviar Black 750GB
Display(s) Hanns G LCD 22" + Acer LCD 22"
Case NZXT Hades Mid Tower Case
Audio Device(s) Realtek ALC892
Power Supply Antec Earthwatts 650W
Mouse Gigabyte M6900
Software Windows 10 Enterprise
#11
The fun starts when you lose your private key due to fuckup, HDD crash or similar. Also trojan on your system could steal the private key (just like a keylogger can steal your typed password).

SSH password logins are perfectly safe and probably 90% of unix systems run with it. Weak passwords are not.

We moved SSH to another port on our servers to get rid of random (chinese) people trying to bruteforce it.
I'm trying to work out how anyone could get my ssh password and I don't think they could, 10 digits long random letters and numbers to anyone but me. More likely is someone stole the key off my pc with a trojan, but still not very likely.

Is there anyway for me to get rid of this thing? What logs would tell me which processes are responsible? Because there must be something running as root that is letting them in now after I have changed the password and key. Btw the server is in a datacenter.
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,056 (3.44/day)
Likes
17,958
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#12
use the "last" command, "top", "ps", check /var/log/messages

documentation for these commands can be found by running "man last" or "man top" etc
 
Joined
Sep 1, 2007
Messages
316 (0.08/day)
Likes
27
Location
UK
System Name Moose 1055
Processor AMD Phenom II X6 1055T
Motherboard ASRock 990FX Extreme3
Cooling Thermaltake PW 850i Liquid Cooler and loads of Case Fans
Memory OCZ (2x2GB) + Crucial (2x4GB) = 12GB DDR3 1600MHz
Video Card(s) Powercolor R9 285
Storage Corsair BX100 250GB + 2 x Western Digital Caviar Black 750GB
Display(s) Hanns G LCD 22" + Acer LCD 22"
Case NZXT Hades Mid Tower Case
Audio Device(s) Realtek ALC892
Power Supply Antec Earthwatts 650W
Mouse Gigabyte M6900
Software Windows 10 Enterprise
#13
Well the good thing is "last" command shows that my ip and :1 are the only ones to login to the server as any user including root for the past month.

Using top and ps x and ps aux, nothing struck me as being an obvious problem except the 2-4 "sshd: root" processes running and the 2-4 "sshd: root@notty" processes running (but apparently neither are being logged into?!)
 

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,056 (3.44/day)
Likes
17,958
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#14
find out what process is sending those emails, find out how it got on your system
 
Joined
Sep 1, 2007
Messages
316 (0.08/day)
Likes
27
Location
UK
System Name Moose 1055
Processor AMD Phenom II X6 1055T
Motherboard ASRock 990FX Extreme3
Cooling Thermaltake PW 850i Liquid Cooler and loads of Case Fans
Memory OCZ (2x2GB) + Crucial (2x4GB) = 12GB DDR3 1600MHz
Video Card(s) Powercolor R9 285
Storage Corsair BX100 250GB + 2 x Western Digital Caviar Black 750GB
Display(s) Hanns G LCD 22" + Acer LCD 22"
Case NZXT Hades Mid Tower Case
Audio Device(s) Realtek ALC892
Power Supply Antec Earthwatts 650W
Mouse Gigabyte M6900
Software Windows 10 Enterprise
#15
Well the process is "sshd: root" it's the one connecting to mail servers.

I have stopped it doing it by changing sshd port, but that isn't a very good fix as they shouldn't be able to do it on any port!
 

qubit

Overclocked quantum bit
Joined
Dec 6, 2007
Messages
14,551 (3.98/day)
Likes
8,058
Location
Quantum Well UK
System Name Quantumville™
Processor Intel Core i7-2700K at stock (hits 5 gees+ easily)
Motherboard Asus P8Z68-V PRO/GEN3
Cooling Noctua NH-D14
Memory 16GB (4 x 4GB Corsair Vengeance DDR3 PC3-12800 C9 1600MHz)
Video Card(s) Zotac GTX 1080 AMP! Extreme Edition
Storage Samsung 850 Pro 256GB | WD Green 4TB
Display(s) BenQ XL2720Z | Asus VG278HE (both 27", 144Hz, 3D Vision 2, 1080p)
Case Cooler Master HAF 922
Audio Device(s) Creative Sound Blaster X-Fi Fatal1ty PCIe
Power Supply Corsair HX 850W v1
Software Windows 10 Pro 64-bit
#16
Does it look to you like the server has been rooted ie running malware? If so, format and reinstall, don't waste your time trying to clean it up.
 
Joined
Sep 1, 2007
Messages
316 (0.08/day)
Likes
27
Location
UK
System Name Moose 1055
Processor AMD Phenom II X6 1055T
Motherboard ASRock 990FX Extreme3
Cooling Thermaltake PW 850i Liquid Cooler and loads of Case Fans
Memory OCZ (2x2GB) + Crucial (2x4GB) = 12GB DDR3 1600MHz
Video Card(s) Powercolor R9 285
Storage Corsair BX100 250GB + 2 x Western Digital Caviar Black 750GB
Display(s) Hanns G LCD 22" + Acer LCD 22"
Case NZXT Hades Mid Tower Case
Audio Device(s) Realtek ALC892
Power Supply Antec Earthwatts 650W
Mouse Gigabyte M6900
Software Windows 10 Enterprise
#17
I reinstalled on a new server new IP helped and was cheaper, took hours of time though.
 

qubit

Overclocked quantum bit
Joined
Dec 6, 2007
Messages
14,551 (3.98/day)
Likes
8,058
Location
Quantum Well UK
System Name Quantumville™
Processor Intel Core i7-2700K at stock (hits 5 gees+ easily)
Motherboard Asus P8Z68-V PRO/GEN3
Cooling Noctua NH-D14
Memory 16GB (4 x 4GB Corsair Vengeance DDR3 PC3-12800 C9 1600MHz)
Video Card(s) Zotac GTX 1080 AMP! Extreme Edition
Storage Samsung 850 Pro 256GB | WD Green 4TB
Display(s) BenQ XL2720Z | Asus VG278HE (both 27", 144Hz, 3D Vision 2, 1080p)
Case Cooler Master HAF 922
Audio Device(s) Creative Sound Blaster X-Fi Fatal1ty PCIe
Power Supply Corsair HX 850W v1
Software Windows 10 Pro 64-bit
#18
Yeah, ya just gotta go clean with it sometimes. I know what you mean about spending hours on it, lol.