I've been running a remote server in the states for many years now, but we are stepping it up. Just upgraded to a 10gb/s connection, and are getting ready to set up websites, gaming and teamspeak servers ect. It won't be too heavy at first. But I need a reliable os to handle the new, probably much higher traffic. Currently running win7 home, which plagues me constantly, I'm trying to convince the owner to upgrade to a server os, but he's complaining about the price. So what can I use that won't cost an arm and a leg that will make my life easier?
Go Linux server if want free and have time to learn it. It is a good way to go but can take a lot of time and headache to achieve the final results one would seek. Debian and Ubuntu Server are good places to start IMHO for those less familiar with Linux server environments. You might check out CentOS as well.
Go Windows Server 2012R2 or 2016 if you want a Windows server OS for a commercial environment. SBS and Home Server versions have restrictions that may or may not be an issue for you in a hosted environment, and generally I've seen more issues with those versions than not. But if you get a good deal or free on a key from above, go for it! It'll still likely be better than hosting those services on a Windows 7 Home installation.
Complaining about the price makes little difference if the owner is expecting enterprise-grade results. If he expects those results, he needs to pay for it, period. That's like buying a Fiat from 1970 and expecting it to run like the brand new La Ferrari.
You can however run Windows Server 2012R2 unactivated using a KMS key from Microsoft (Google "Microsoft KMS Key"), and it will allow operation and OS security updates. You will be somewhat limited on other product updates, and if your business faces a Microsoft audit, will be fined for not having a license or any CAL's for the environment you're in.
There are some registry tweaks you can make to Windows 7 (I believe Pro) that can allow it to be a better file server. Frankly I'd go with Server 2012R2, it's stable, aging well, and widely supported still.
I run 2012R2 Datacenter (key from my College IT courses years ago), and have about 8-12 VM's. I have Plex, TeamSpeak3, several Minecraft servers in one MineOS VM, website, OpenVPN, PFSense, WDS test VM's, etc. on various deployments. Some VM's are Server 2012R2, some are Linux Server. A good way to go is to setup Hyper-V and run your actual server that hosts your services as VM, then you can backup the VM to a NAS or at least USB hard drive, so if the physical "core" server kicks the bucket, you can spool up the VM on another physical server and get back up and running much faster.
You don't need a separate VM for each task or hosted situation, it might not hurt to split some of them up. I could run 3-5VM's in my lab environment to do all the things I do with 8-12.
Do you plan to host a domain environment as well?