1. Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Best way to set up a huge 10~20TB for camera surveillance?

Discussion in 'General Hardware' started by n-ster, Mar 16, 2012.

  1. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    I will be building a computer soon. This computer's only purpose will be camera surveillance which will be running 24/7. We need a lot of storage, at least 10TB if not more, as this will have 4x 3MP cameras (2048x1536@15fps) and 12x 2MP cameras (1080p@30fps), which would mean 10TB should cover a month (300GB/day on average)

    What type of drives should I use to have the best endurance? Reliability isn't crucial, but bang/$$ is. Recommendations for very long lasting PSUs that work great 24/7 would be good too (should I be going into server PSUs?). Would Ivy Bridge do or is X79 better?

    This will be running Windows 7, probably ultimate 64 bit of course, is there anything I should be worried about when building a comp with such a huge amount of disk space?

    Can I just use the hard drives separately instead of RAIDing?
     
  2. mudkip

    mudkip

    Joined:
    Dec 14, 2008
    Messages:
    1,226 (0.57/day)
    Thanks Received:
    148
    Location:
    The Netherlands
    :wtf:
     
    yogurt_21 says thanks.
  3. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,945 (3.92/day)
    Thanks Received:
    11,735
    10 or 20 TB is far from huge for today's standards.

    google: unraid
    if you can use freebsd or solaris: ZFS
    if you have windows server, look into DFS Distributed File System

    what sustained read/write bandwidth is needed for your cams ? i assume they all write to the same storage?

    what about transcoding the video you get?
     
    n-ster says thanks.
  4. CJCerny

    CJCerny

    Joined:
    Nov 25, 2008
    Messages:
    855 (0.40/day)
    Thanks Received:
    249
    Location:
    Akron, OH
    Seems to me like the reliabililty of your hard drives and PSU are the least of your worries. You main worry needs to be how you are going to get all that data from those cameras to this PC. If they are all IP cameras, you are going to need to make sure that you can handle all the network traffic that they are creating. If they are USB cameras, you're going to have a hard time building a PC that doesn't drop frames with all that input via USB.
     
  5. ShiBDiB

    ShiBDiB

    Joined:
    Jul 21, 2008
    Messages:
    4,087 (1.79/day)
    Thanks Received:
    767
    Location:
    Clifton Park, NY
    main reason I could see to RAID would be for a backup. Drives constantly writing like these will be, is gonna put them under alot of stress.

    Might be better in the long run to look to a professional company for something of this scale.
     
    n-ster says thanks.
  6. v12dock

    v12dock

    Joined:
    Dec 18, 2008
    Messages:
    1,587 (0.74/day)
    Thanks Received:
    317
    iSCSI SAN
     
    n-ster says thanks.
  7. AnomalouS

    AnomalouS

    Joined:
    Jun 19, 2008
    Messages:
    350 (0.15/day)
    Thanks Received:
    39
    Location:
    Lincoln, NE
    Must you keep the captured video for a month at a time? why not use first in first out method.
    In the medical world there are capture devices that do just the FIFO, xxxGB drive that purges data as needed.
     
    n-ster says thanks.
  8. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    Haha, it is good to have but if we loose 1 HDD, that would mean about a week and we can handle that every couple years or something. I'm not going to pay 3~4x the price for 1.5x~2x the longevity for the HDDs is what I meant

    It's huge for me :p I'll look into all that in a few minutes, thanks.

    The cameras are IP and wired with ethernet cables. The max they would use is 8Mbps which is 1MB/s per camera, so about 16MB/s I'm guessing is the peak rate. The coding of the video is already MJPEG or H.264 and the camera's processors handle that. It'll be coming through 1 Ethernet port

    The way it our current system works: say there is 5x 2TB HDDs. It will write on the first drive, once that is full it will start on the 2nd drive and so on, once all 5 are done it will start overwriting the first drive again.

    They are going to be wired with Ethernet cables, which will go to a 24 port Ethernet switch thing (each camera will have their own 100 Mbps cable/port) and exit through 1x 1Gbps Ethernet cable.

    I believe that is how we are doing it. At any time we look at the data, the oldest we will be able to find is 1 month old. Our old system writes on each HDD until they are all full then start overwriting on the first drive etc.


    We did in the past and they charge you so much, and they end up doing it worse than I would have. They charge 3800$ for a i5-2400 with the IGP and some cheap chipset and 8GB of RAM and 4TB with inadequate cooling and a very average PSU with 1 year warranty. They user WD caviar black HDDs, nothing fancy.

    I'll have to google for specifics it looks interesting
     
    Last edited: Mar 17, 2012
  9. jsfitz54

    jsfitz54

    Joined:
    Jun 18, 2010
    Messages:
    903 (0.57/day)
    Thanks Received:
    242
    Not all months are equal.

    Would a 45 day first in first out method be better?

    If for some reason you need legal copies, in a 30 day time frame, you would have extra days to recover the necessary footage before overwriting occured.
     
  10. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    the 30 day thing is just an estimate, if 1 month only takes 6.6TB than I can look back about a month and a half behind etc. After the first time the 10tb is filled, all it will do is overwrite the oldest videos.

    I'm wondering if I should just do one of those RAID 5 boxes or setup a RAID 5 inside the computer, it would be more reliable and faster. I'd probably need a RAID card though? unRAID looks good too

    As for the PSU, am I best going with a Platinum PSU or looking into server type PSUs? Remember this will be on 24/7 365 days

    I'll have to use Windows 7 Ultimate, so linux distros etc are out of the question.

    can anyone help me understand iSCSI SAN better? What do I need etc.
     
  11. Kreij

    Kreij Senior Monkey Moderator Staff Member

    Joined:
    Feb 6, 2007
    Messages:
    13,881 (4.93/day)
    Thanks Received:
    5,616
    Location:
    Cheeseland (Wisconsin, USA)
    How flexible is the software for the surveillance system?

    If it will allow you to stream the video directly to multiple NAS devices, I would make a large RAID5 NAS device with hotswap disks. You could then add more NAS devices if the storage requirements grow.
    You could then slap any old HDD (or a pair in RAID1 for redundancy) in the computer for managing the setup and viewing the video.

    It would not be the cheapest way to go (good/fast NAS devices aren't cheap), but it would add some robustness and future proofing to the configuration.

    If the software will not allow direct streaming to a NAS, you would put in a pair of gigabit NICs (one incoming, one outgoing) and let the computer forward the data to a single large NAS device.

    Just some some thoughts. :)
     
    n-ster says thanks.
  12. yogurt_21

    yogurt_21

    Joined:
    Feb 18, 2006
    Messages:
    4,418 (1.39/day)
    Thanks Received:
    575
    Location:
    AZ
    based on the amount of data it seems hard disk performance is far more important cpu/mobo/memory performance.

    personally i'd look into some cheaper perhaps older spec server hardware for the longevity. Then throw most of the money into hard drives.

    specifically if you could find a used Dell PowerEdge 2900 it would have everything you would need while being a bit slow by today's standards in cpu/memory.

    The chasis supports 8 hotswap disks and a couple more internal as well as a few 5.25's fro whatever your i/o needs are.

    it has dual psu's for redundancy and should be very reliable (but again not the fastest thing in the world)

    The only upgrade needed would likely be 2 tb drives for the hot swaps and a gpu upgrade for any playback needs. The raid controllers should still be fine.

    They currently go for 600-1000$ on the used market, compared with psu/tower that supports enough hard disks, mobo/memory/cpu, and a raid controller it would be hard to beat and Way, Way, Way more reliable than desktop hardware. Remember desktop hardware is only designed to be on 8 hours a day and typically 3 years longevity (longer for cpu's/memory on average though) but that means in 1 year under 24/7 use you will have used up the full life of most of the hardware.
     
    n-ster says thanks.
  13. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    Used stuff is pretty much out of the question. If I have to assemble a server, that's what I'll do. But does, say, a Xeon CPU really outlive a standard i7? My plan was just to wait until Ivy Bridge and build them that with a server or platinum PSU and good hard drives and plenty of fans (so that if a few fans fail it won't change much). I was also just going to go with a big full tower case. Should I do it differently?
     
  14. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,945 (3.92/day)
    Thanks Received:
    11,735
    it's the same silicon, made in the same production process, in a different package

    dont worry about ivy bridge. dont worry about good hard drives.

    get something like ASRock P67 Extreme6 (because it has 10 sata ports or so). add more sata ports using cheap controller cards. 2500k (K because it's only 5 bucks more expensive)

    fill up with the cheapest hdds you can find (wd caviar green 2.5 tb). get some raid config setup so you dont lose your data

    windows 8 will have storage pools which should make your life much easier
     
    Last edited: Mar 19, 2012
    theoneandonlymrk and n-ster say thanks.
  15. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    Sounds great except for the WD Green HDDs don't have the greatest reputation... Are the problems fixed with them?

    Also would the stand intel chipset be OK for the 10~20TB RAID or should I be going with a RAID card? And 10 SATA ports should be enough, so I wouldn't need the controller cards?

    I'm going Ivy Bridge as that is when I'll be building this anyways... The earliest I'll be building this is after my Finals, which would mean May, which is when IB is coming out right? If it isn't out by the time I actually build this, I'll go 2500K as you suggest, but I would assume the IB will have lower consumption, which would likely help with longevity and will save a few bucks at 24/7 use.
     
  16. W1zzard

    W1zzard Administrator Staff Member

    Joined:
    May 14, 2004
    Messages:
    14,945 (3.92/day)
    Thanks Received:
    11,735
    you should really have some sort of raid to prevent data loss in case of disk failure.

    once you have that it doesn't matter whether you use expensive disks that fail once in 5 years or cheap ones that cost half as much and fail in 2 years and get replaced for free thanks to RMA

    out of those 10 ports only 4 or 6 are on intel. makes no difference anyway. personally i've had better experience with software raid than with hardware raid
     
    theoneandonlymrk and n-ster say thanks.
  17. yogurt_21

    yogurt_21

    Joined:
    Feb 18, 2006
    Messages:
    4,418 (1.39/day)
    Thanks Received:
    575
    Location:
    AZ
    like I said cpus and memory are typically the exception. psu, mobo, etc not so much. But W1z is right xeon vs regular is essentially the same cpu. When it comes to failing parts for the 2000 servers I'm responsible for (they sit behind cleint networks and house media content locally rather than sucking bandwidth) 1. hard disks 2. motherboards 3. raid controllers 4.memory. I've yet to have a cpu fail on me since Ive been in this position (4 years) that includes pentium grade up to highend xeon.

    have had plenty of hard disks and to a slightly lesser extent motherboards. Most the boards that have fried were on our lowend models using desktop hardware. I have only had a couple server boards go out over the course of 4 years.

    the only issue with going brand new is that case alone will net you half of an old server cost. You need something that houses enough hard disks and will still be effiencient for cable management and cooling.

    whatever you do don't cheap out on the raid controller, you'll quickly find managing alot of data on a poor controller = hell on earth.
     
    n-ster says thanks.
  18. v12dock

    v12dock

    Joined:
    Dec 18, 2008
    Messages:
    1,587 (0.74/day)
    Thanks Received:
    317
    Hardware RAID is much more scalable and it will allow a lot more option for setting up RAIDs

    Also are you dealing with raw data or will there be any encoding?
     
    n-ster says thanks.
  19. phanbuey

    phanbuey

    Joined:
    Nov 13, 2007
    Messages:
    5,205 (2.05/day)
    Thanks Received:
    975
    Location:
    Miami
    n-ster says thanks.
  20. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    Motherboards actually have that high of a failure rate? I didn't know. But server or Workstation mobos seem really expensive comparatively to standard mobos... What do I have to look for to get a lasting MB?

    For the case, can't a standard large Full Tower case do that job?

    Memory is the least of my worries as I have plenty and if some craps out ill just pop in some other sticks pretty easily. PSU I'll be going something like the LZP-550W from Enermax

    The data will basically be H.264 video files and a few MJPEG video files
     
  21. phanbuey

    phanbuey

    Joined:
    Nov 13, 2007
    Messages:
    5,205 (2.05/day)
    Thanks Received:
    975
    Location:
    Miami
    If you really need that much storage, DO NOT tie it down to one machine. You want the storage dtachable so that if the server goes you can restore image to another server and hook it up to the data.

    A single workstation is usually not a good way to go (for something like this which is essentially a video server). You're better off building a super cheap server with detachable storage. Just my 2c. Also detachable storage usually has its own monitoring and alerting tools, which will make your life alot easier.
     
    n-ster says thanks.
  22. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    So basically a RAID 5 box?

    So that 1 ethernet that carries the video files goes to the computer, and I use the RAID 5 box as the HDD where it saves the video files and be done with it right?
     
  23. phanbuey

    phanbuey

    Joined:
    Nov 13, 2007
    Messages:
    5,205 (2.05/day)
    Thanks Received:
    975
    Location:
    Miami
    That is the cheapest storage we trust. I only picked that one because we use that same one in one of our diagnostic centers for Radiology images, and it works pretty good. It basically serves the exact same purpose which you need - cheap, redundant and lots of it.

    ^yes basically exactly right - the raidpack attaches through eSATA and the OS and Computer see it as a gigantic hard drive.

    This one is even cheaper (although you will need software raid):

    http://www.high-rely.com/hr_66/products/hr-rack-mount/
     
    Last edited: Mar 19, 2012
    n-ster says thanks.
  24. v12dock

    v12dock

    Joined:
    Dec 18, 2008
    Messages:
    1,587 (0.74/day)
    Thanks Received:
    317
    Could possibly setup a RAID 10 either way I would look into getting a nice raid controller

    What kind of hard drives are you looking at?
     
  25. n-ster

    Joined:
    Jan 11, 2009
    Messages:
    8,859 (4.20/day)
    Thanks Received:
    1,405
    I was going to get WD Blacks but I'm not sure anymore lol
     

Currently Active Users Viewing This Thread: 1 (0 members and 1 guest)

Share This Page