looking for help/advice on how to log/record sites/url's visited on home net

[jboydgolfer]

HEllo TPU,

as my title says, im looking for a way to monitor the sites visited on my local home network.
i have 3 children here, and even though i have spoken with them regarding "smart surfing" and i really do trust them, i would like the ability to have a fall back, since the middle boy 12y/o got a little out of control last year with sites his mother and myself would not approve of.
My old Netgear WNDR3800 did this to some degree, whether it was a DNS log, or what i dont know, but it gave a url list, that was emailed to me every day. My new Asus RT-AC66W(ac66u) running the current latest verson of merlin seems to lack this ability, so i am looking for a way to do it on this new router.**i HAVE gone through this merlin interface, it claims to have loging, but it isnt implemented, or is useless.**
i know a small amount when it comes to networking, id say more than the average joe, but im certainly NO expert. any constructive help would be VERY appreciated. i already have an openDNS account, but this isnt available thru them either, they only have DNS logging.

thanks.

My current router.
Asus RT-AC66W(AC66u) in white.

 

[remixedcat]

You can possibly use prtg and add the packet sniffing sensor

If not you may want to run a proxy server that has logging

Squid is a good one to start out with

 

[Kursah]

+1 Squid works well...proxy servers that feature logging can be made for fairly inexpensive or be an old PC...I use my pfSense box to run the Squid service on it as well. It can be a little unwieldy to read through the massive logs Squid can create depending on what you use to access said logs...but it's not too shabby.

http://www.squid-cache.org/

I have been told Untangled offers better options in this regard...I haven't tested it for proxy services but apparently my coworker swears by it.

Being an OpenDNS user you should be able to see site reports, but as you found it'll only be from your public IP and not your local network, but the end results are the same as you will know your site is trying to access. Nice thing is you can make a global list of allow/deny with OpenDNS if you know that noone at your location is needing to access said site. You make your router give OpenDNS DNS servers out through DHCP. Then say you want your PC to not have those restrictions, use the ISP's DNS or Google DNS and you're good-to-go.

I guess it depends on if you need to know what your clients are viewing or if you need to filter against what you don't necessarily want them to view from your Internet connection.

Another option, that would be more cumbersome but fast to deploy would be running Wireshark over your network, you would be able to view, sort and see what packets are going where which will contain requests. Filter for port 80 primarily to see website requests.

Frankly I think Squid is the best way to go initially. Though I've been meaning to find a better reporter plugin for pfSense so that my log sorting is easier...I can attest that it does its job very very well and is free.

 

[jboydgolfer]

thank you both.

 

[remixedcat]

You're welcome hope it goes well

 

[qubit]

Squid works well and there are loads of other ways to do this too.

I run the IPCop firewall which has got a proxy server on it. It logs every single URL if you want it too as well and the log is easy to browse through, with an export function if you want to do further data analysis on it. Your kids are gonna hate you.

It's free, based on Linux, takes over a whole PC, is very secure, easy to set up and works very well. There's other similar Linux firewalls out there too, like the original SmoothWall that it forked off years ago. There's also Monowall too. I think you should try IPCop as a starting point though, because it's so easy and secure.

www.ipcop.org

 

[jboydgolfer]

Squid works well and there are loads of other ways to do this too.

I run the IPCop firewall which has got a proxy server on it. It logs every single URL if you want it too as well and the log is easy to browse through, with an export function if you want to do further data analysis on it. Your kids are gonna hate you.

It's free, based on Linux, takes over a whole PC, is very secure, easy to set up and works very well. There's other similar Linux firewalls out there too, like the original SmoothWall that it forked off years ago. There's also Monowall too. I think you should try IPCop as a starting point though, because it's so easy and secure.

www.ipcop.org

ill try that ipcop, as im hoping to do this with minimal effort. thanks again everyone
is this a program i can run on My PC, or does it require a machine of its own?
*edit* it seems as tho it Does require its own machine, sadly i dont have one to spare currently, ill needs to keep looking

 

[Ahhzz]

I setup a Squid proxy several years ago for our business, my first foray in the linux world. It was a bit of a learning experience, but it absolutely will do what you want out of it...

 

[hat]

I wonder if another firmware could offer such functionality? The RT-N66U is very flexible.

 

[jboydgolfer]

I wonder if another firmware could offer such functionality? The RT-N66U is very flexible.

i agree.
i never really used the stock firmware all too much, but the Merlin interface is pretty similar , but just builds onto what is already there in both appearance, and functionality.
My old Netgear WNDR 3800 did traffic monitoring quite well, iirc, what it does is called "DNS monitoring" but if someone went to "choke my wife while i throw bologna slices at your ass . com"
i knew about it, and thats all i really want. i dont need to know EXACTLY what they are doing on the sites in question, i just need to know if they are going to sites i dont approve of is all.

 

[qubit]

ill try that ipcop, as im hoping to do this with minimal effort. thanks again everyone
is this a program i can run on My PC, or does it require a machine of its own?
*edit* it seems as tho it Does require its own machine, sadly i dont have one to spare currently, ill needs to keep looking

Ok, shame. You can run it in a VM, but that would defeat the purpose.

 

[jboydgolfer]

Ok, shame. You can run it in a VM, but that would defeat the purpose.

i appreciate the suggestion just the same.
i looked into the program on theyre site, and its everything you said, Very good recommendation.

 

[Kursah]

You might try DD-WRT. I am pretty sure some flavors of it support this...and the AC66 series should be able to handle it. Might be your ticket.

 

[hat]

"choke my wife while i throw bologna slices at your ass . com"

Oh god that's fucking hilarious...

Anyway, maybe try the different firmware builds out there. I'd be surprised if Tomato couldn't. I have Tomato on mine and the options are very extensive, though I've never looked for that functionality so not 100% sure. If not, it should be easy to get your hands on some bullshit PC you could run ipcop or something on. Even a Pentium 2 could do that. Then, if you find you like it and are serious about running it, if you wanted to you could look at more efficient hardware.

Still, I'd be surprised if at least one of the third party options didn't offer this for your router.

 

[jboydgolfer]

Oh god that's fucking hilarious...

Anyway, maybe try the different firmware builds out there. I'd be surprised if Tomato couldn't. I have Tomato on mine and the options are very extensive, though I've never looked for that functionality so not 100% sure. If not, it should be easy to get your hands on some bullshit PC you could run ipcop or something on. Even a Pentium 2 could do that. Then, if you find you like it and are serious about running it, if you wanted to you could look at more efficient hardware.

Still, I'd be surprised if at least one of the third party options didn't offer this for your router.

there is some worthless "Traffic monitor" built into Merlin, but Merlin himself posted that the kernel or whatever it is called IS present in the merlin Firmware, but he never implemented it.

i have an i3 4160, id just need a Mobo , and RAM,

 

[hat]

Well, get the cheapest ram you can possibly find. You could literally do this with 128mb PC133 or less, but such ram isn't compatible with a Haswell system. I don't know if Haswell power saving features work in software environments like IPCop though...

 

[jboydgolfer]

Well, get the cheapest ram you can possibly find. You could literally do this with 128mb PC133 or less, but such ram isn't compatible with a Haswell system. I don't know if Haswell power saving features work in software environments like IPCop though...

eh, the power, or the cost of building the PC is no issue, mainly it would be the justifying it to the woman i sleep next to

 

[Kursah]

That CPU is overkill but will work great. I'd say sell it. Buy an Asus N3150-C (I've ran these for routers, NASs, VM servers, media servers)...super power efficient. Get 2-8GB DDR3...depending on your needs...2-4 is likely more than enough. Get a cheap 120GB SSD since you don't save enough to justify a 60GB.

I used a case that came with an SFX PSU. Runs my pfSense at home with room to spare. I used a dual port gigabit Intel NIC I got on fleabay for $30 shipped.

With pfSense you'll enjoy Intel comparability for usage, thermal sensor reading and encryption acceleration. Add Squid as a package...give it 10GB-15GB bor caching on the SSD and add a UI plugin to review the logs.

You'll be able to use your AC66 in AP mode and have a real router. pfSense runs circles around home grade routers. My pfSense box is absolutely excellent. I can link you the parts I used when I get home...in kept it cheap. Maybe not what you wanna do...but sell that i3 and you'll have paid for at least half.

IPCop is okay as well...but given the choice I'd go pfSense all day long. Both are good and so is Untangled. But pfSense wins in my book.

 

[hat]

I ran dd-wrt x86 on an old pentium 4 box I received from my school once. I ran it off of a CF card, 128MB ram, and an old WIllamette 1.3GHz if I'm not mistaken. I slapped in a network card for my LAN interface (onboard ethernet was WAN), ran that to a switch which had a wireless AP connected to it, as well as my hardwired devices.

It was an interesting project, but in the end I decided it was a frankenstien'd version of what a basic consumer class router could do much more efficiently, in many ways. I think after that I wound up with a Linksys E1000, which served me well, but was ultimately left behind in my life. Now I have the RT-N66R and I have zero plans for a new router.

 

[Kursah]

@jboydgolfer

Here's my pfSense build:

• Board/CPU (SoC) - Asus N3150-C w/Quad Core Celeron - $82 @ Amazon & Newegg
• 8GB DDR3 1600 - 2x4GB Crucial Ballistix Sport CL9 - $32 @ Amazon
• 120GB SSD - A-Data Premier 120GB SSD - $39 @ Amazon
• Intel PRO/1000PT PCI-e 2-port Gigabit NIC w/half height and full height brackets - $28 @ Ebay from Serveronics
• In-Win BP655 Case w/300W PSU and USB 3.0 (rated better than the $15 cheaper 200W combo by far) - $69 @ Newegg

TOTAL: $250

Now I beefed up my storage and memory as I wanted a fast SSD and 8GB DDR3 so I could play and test some higher-end enterprise-grade filtering, caching, proxy, snort/IDS, and have the ability to change it over to a Server 2012R2 Hyper-V host at some point if I decide to repurpose my current server. The N3150 Celeron is an excellent little CPU, runs Windows and Linux great, that Asus board is solid and cool running with barely any airflow I hit 40-42C max. It has a single PCI-e 4X slot perfect for the added NIC...as the integrated Realtek is not recognized by pfSense directly...it would be fine if given a virtual NIC through a VM though as Windows has no such issues nor do other flavors of Linux...pfSense being based on slightly older BSD doesn't recognize many Realtek NIC's.

This board/SoC combo is plenty for many types of projects, a NAS, router, a server that runs both as VM's, a dedicated or virtual media server (Plex, Kodi, etc)., you name it...all while consuming around 15W under load iirc. My old Ubiquity EdgeRouter Lite-3 consumed approx 7W and has a fraction of the power and memory that this build does...sure it cost $100 for a total package...but required command-line to be able to properly configure all the settings needed...where pfSense is I only used the GUI. Though I don't mind CLI.

The Intel PRO/1000PT NIC is a damn fine performer, seen in all OSes straight away, excellent performance, beats all other add-in NIC cards I have in my inventory (which is mostly a bunch of cheap Realtek...though many that cost more than this NIC!!!!). For $28 you won't find better IMHO.

The case can also be changed...I spent a lotta time...I was going to get a $50 mITX case off of another site that was bare bones, slap in a 150W PICO-PSU and call it a day...but it came out slightly more expensive since I didn't have a suitable laptop power brick. But the BP655 is still small enough...just not the smallest. I did take out the drive cage. I screwed the SSD into the front grille and had plenty of room for cables, mild management and was good-to-go. I can upload some pics if you are interested. I also did not use the included case fan...it runs 100% balls out and is server-grade loud...so I rely on the super quiet and low airflow PSU fan, and had 0 issues at all. Super quiet and efficient. The 300W PSU I believe is a different supplier and from what I read is of far better quality than the 200W version...so to me that was worth the extra $15 up front. Again 0 issues thus far..

Do you need 8GB of RAM? No...hell I barely use 2GB with all the shit I'm running, which is IPSec and OpenVPN tunnels, Squid, ClamAV, and all sorts of junk...but I also have a lot left to test and again wanted the spare room. My pfSense VM runs perfect on .5-1GB of RAM and 1 CPU core with no complaints and no noticeable drop in speed until VPN tunnels are involved and the encryption overhead starts to take a toll. I wanted full speed WAN VPN...and seeing I only have 60Mbps...the 3150 can handle far far more than I'll likely get to my area for the next 5 years. Some claiming it can handle up to 800Mbps or even a gig throughput over an encrypted VPN tunnel...I cannot verify those claims, just what I read. frankly at 60 it's excellent and very low loads on the CPU.

The SSD is of decent quality and I've used a lot of em for various projects...not the cheapest, not the most expensive, and a solid performer that sips power and has a decent warranty to boot. For this job it is beyond excellent..keep the power down, cheap costs, MORE than enough for this task...easy to mount and it runs cool. For the price, it is sure tough to beat with anything of quality IMHO...though with prices changing all the time...you might find better deals new/used if you look around that make this look like junk. Never know.

So toss the right version of pfSense on a USB flash drive, boot to it, install, choose a WAN and LAN port (it helps you and usually gets it right as well..I go top WAN and bottom LAN). There's tons of documentation and videos to help you, along with a few knowledgable users here that could help you get it setup.

You'll have a more stable and secure network, with more control and ability and a router you could turn into a PC if needed. Save the current config on your Asus router so if you need to revert it to router-status, all you have to do is reload the config...then once that's done, set it up as an Access Point and it'll feed wireless from wherever you want...so if there's a better place to install it in your house go ahead, as-long-as you have an Ethernet cable to run to it's WAN port, you're golden. Another nice thing is you'll have those 4 ports to still use as well, so it's an AP + 4-port switch.

You'll have a better firewall, more stat tracking abilities, 1000s of plugins and modules you can install to mess with for security, monitoring, filtering, connecting, etc, etc etc. Usually pfSense itself + Squid is plenty for most.

I work with and deploy dozens of Netgate and pfSense (Netgate) firewalls, usually the SG2440, which costs about 2X what the build above does with about 1/3 of the performance/hardware of my build. It's a lot smaller and its purpose is as a dedicated router...honestly where my router sits I don't notice it anymore. It is quiet and stable. I run pfSense 2.3 and couldn't be happier.

Again, might not be the right choice for you...but it is an option. I will say my network is stable, fast, efficient, and I don't have issues...between Squid, OpenDNS and pfSense's firewall (it's really just Linux IPTables), my network is in good shape. Should it die, my EdgeRouter Lite is ready to drop back in, should that die, my AC66R is ready to drop back in. Failovers are a good idea to keep the wife and kids happy! Plus if a PC dies, I can have this box re-purposed in under an hour. Really plenty of good reasons to build and play with a dedicated pfSense/router box, should you want to. That's what it comes down to...and for me, totally worth it. For many, too much work...for you, I think you could do it and I also think you'd enjoy pfSense, IPCop and/or Untangled.

TL;DR

Sell your CPU for $100, spend $100-150, have a far better router, that can be used as a PC/Server as well if you chose...use the AC66 as an access point. You'll have options should it ever fail or need re-purposed and it sips power. Win-win.

You could also really budget this to death, and go with older Core2/DDR2 parts without issue...I've even temporarily deployed laptops as routers before.

 

[remixedcat]

eh, the power, or the cost of building the PC is no issue, mainly it would be the justifying it to the woman i sleep next to

Hell If I was your girl I'd be all for it to save nerves and crap lol.... Hell I'm the woman of the house and I dick w hardware more then the hubs does!!!

 

[SnakeDoctor]

PF SENSE - FTW

User friendly ,
all packages easily available with one click,
Low hardware requirements

Transparent proxy with squid ,ntop, Bandwidthd
Ntop - gives a nice gui of all pc on network can see sites each ip visited
Bandwidthd - usage tracker

 

[jboydgolfer]

so maybe someone here can answer this additional "situation" i have encountered since last night.

I was in my garage, and i logged onto the Router Web UI with an iphone, and its obviously no different, as one would expect (or so i thought) except, options are available on iphone , that Arent on My PC!! the same goes for android too, ONLY the IOS devices, see these other option, as where PC, and android do not.
look , and see for yourselves. clearly, only 2 options are available for PC, and Android in the UI, but when I log into the exact same UI with Iphone, i get additional QOS options. WTF?!
PC:

Android:

Iphone: **sorry its blurry, my tablets camera had oil or some goo on it.**
also, bandwidth limiter was available too, it just uses a separate drop down.

 

[remixedcat]

damn apple bias!

 

[Dethroy]

That's hilarious @jboydgolfer