• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

10Gb network/connection

Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Ok, I need somebody that is more into 10Gb networking than I am. Right now, all of my computers are on a gigabit network.
What I would need is to connect my main PC to my old storage server with something faster than gigabit connection.
The reason behind that is I moved my SAS tape drive to my storage server and would like to do tape backups directly via network (without using HDD in the storage server). For the old LTO-2 tape drive it's not a problem since it's slow, but the newer LTO-4 needs a transfer speed of 80 to 120 MB/s - and that old onboard gigabit NIC cannot keep up - not to mention if I want to run both tape drives at once.

So the best solution for me would be to install two 10Gb NIC (one in the server and one in my PC), and connect them directly. (I would prefer SFP+).
I was looking at older used server 10Gb NIC, like HP NC523 or NC530 which are cheap and easy to get from ebay. But before I buy anything, would my setup even work, due to direct connection?
Also which cable I would need to use since I see that there are different SFP cables (the required distance is around 3m)?
 

Solaris17

Dainty Moderator
Staff member
Joined
Aug 16, 2005
Messages
21,689 (3.97/day)
Location
Florida
System Name Venslar
Processor I9 7980XE
Motherboard MSI x299 Tomahawk Arctic
Cooling EK Custom
Memory 32GB Corsair DDR4 3000mhz
Video Card(s) Nvidia Titan RTX
Storage 1x 250GB 960 EVO | 1x 500gb Intel 720p | 32TB SAN
Display(s) 3x AOC Q2577PWQ (2k IPS)
Case Inwin 303 White (Thermaltake Ring 120mm Purple accent)
Audio Device(s) Schiit Fulla 3 on Beyerdynamic DT 990 Pros
Power Supply Seasonic 1050W Snow
Mouse Roccat Kone Aimo White
Keyboard Ducky Shine 6 Snow White
Software Windows 10 x64 Pro
Dont go with the HP cards, they are Qlogic chips and have terrible driver support and quirks.

Go with mellonox connextX 2 or 3 cards. They are just as cheap.

Next you need to decide how to connect them. Or rather what you are willing too do. 10G RJ45 transceivers via Cat6? fiber? DAC cable?

Finally, if it works all depends on your software. Its totally possible on any OS to assign two different addresses to a NIC. I do that on my servers. Thats how my 10GB network speaks. Using 10. and 172.
 
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Ok, will check those out, if they are cheap in EU as well - hopefully yes.

If I use DAC cables, I don't need transceivers or do I? I'm trying to keep it simple and withing a reasonable budget.

My PC is running Win 10 and on the server it's Win Server 2008 R2, if the backup software will work like it should I cannot know - but I think there should not be problems.
 

Solaris17

Dainty Moderator
Staff member
Joined
Aug 16, 2005
Messages
21,689 (3.97/day)
Location
Florida
System Name Venslar
Processor I9 7980XE
Motherboard MSI x299 Tomahawk Arctic
Cooling EK Custom
Memory 32GB Corsair DDR4 3000mhz
Video Card(s) Nvidia Titan RTX
Storage 1x 250GB 960 EVO | 1x 500gb Intel 720p | 32TB SAN
Display(s) 3x AOC Q2577PWQ (2k IPS)
Case Inwin 303 White (Thermaltake Ring 120mm Purple accent)
Audio Device(s) Schiit Fulla 3 on Beyerdynamic DT 990 Pros
Power Supply Seasonic 1050W Snow
Mouse Roccat Kone Aimo White
Keyboard Ducky Shine 6 Snow White
Software Windows 10 x64 Pro
DirectAttachCables have the transceiver built
In.
 
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Checked the Mellanox cards and they are around 75€ vs 22€ for the HP - but I also saw reports of the HP running really hot - not a problem in the server due to plenty of airflow but it is a problem in my PC.
But I checked what seems to be a newer model the 530SFP and that one is at 9W which is ok, but it's still using qlogic chip. Is there any other alternative which is proven to have good drivers and support?
 

Solaris17

Dainty Moderator
Staff member
Joined
Aug 16, 2005
Messages
21,689 (3.97/day)
Location
Florida
System Name Venslar
Processor I9 7980XE
Motherboard MSI x299 Tomahawk Arctic
Cooling EK Custom
Memory 32GB Corsair DDR4 3000mhz
Video Card(s) Nvidia Titan RTX
Storage 1x 250GB 960 EVO | 1x 500gb Intel 720p | 32TB SAN
Display(s) 3x AOC Q2577PWQ (2k IPS)
Case Inwin 303 White (Thermaltake Ring 120mm Purple accent)
Audio Device(s) Schiit Fulla 3 on Beyerdynamic DT 990 Pros
Power Supply Seasonic 1050W Snow
Mouse Roccat Kone Aimo White
Keyboard Ducky Shine 6 Snow White
Software Windows 10 x64 Pro
Intel or chelsio

see if there are any Broadcom chips available to you, that’s what mellonox runs on.
 
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Found HP 560SFP for 50€ each, and that one is running a intel controller so I guess it should be better than the qlogic chips. Don't know about the thermals/power consumption.

Anyway after a good sleep over my issue I remembered I don't have any pci-e x8 slots free on my PC motherboard. All I have left is the two x1 slots (which would have enough bandwidth for my use case but I won't be able to use the card to it's full potential. I don't even know if the card will even work in an x1 slot.
 
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Just to report back, bought 2x Intel x520-DA2 cards. Got them for the same price as HP 560sfp (it's the same NIC just HP branded).
Also got a DAC cable incoming, hopefully everything will be here by the end of the week.

I just hope that the CPU in my old server will be fast enough to sustain speeds for my tape drive, otherwise I'll have to design an enclosure for the tape drive to attach it to a more modern DL380G6 that I have.
 
Joined
Aug 13, 2009
Messages
2,673 (0.67/day)
Location
Czech republic
Processor Core i7 3770K
Motherboard Gigabyte Z77X-UD3H
Memory 16GB
Video Card(s) Sapphire Radeon Rx 580 Nitro+ 8GB
Storage Samsung 850 PRO 256GB + Samsung 970 PRO 1TB
Display(s) Dell U2415
Case Fractal Design Define R6 Black
Audio Device(s) Creative Sound Blaster ZxR
Power Supply Seasonic PRIME Ultra 650W Gold
Mouse Roccat Kone EMP
Software Windows 10 x64
I am checking Ebay for good deal on either Chelsio T520(-SO)-CR, Mellanox ConnectX-4 or Intel T550 (keep dreaming, these are expensive as hell) semi-regularly myself.
Of course this would be a pricey network upgrade as I need a switch as well and another 10Gb card for my PC (screw the rest of the house, I'm impatient and need speed bump the most :D)
 
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Yes some options even used are still quite expensive, also I didn't even look at RJ45 NICs due to higher price and much higher power consumption/heat output.
Much cheaper to go with sfp+ (used sfp NICs are all server offloads so there is ton of them for cheap).
Bought the cable new from fs.com with Intel coding.
 
Joined
Aug 13, 2009
Messages
2,673 (0.67/day)
Location
Czech republic
Processor Core i7 3770K
Motherboard Gigabyte Z77X-UD3H
Memory 16GB
Video Card(s) Sapphire Radeon Rx 580 Nitro+ 8GB
Storage Samsung 850 PRO 256GB + Samsung 970 PRO 1TB
Display(s) Dell U2415
Case Fractal Design Define R6 Black
Audio Device(s) Creative Sound Blaster ZxR
Power Supply Seasonic PRIME Ultra 650W Gold
Mouse Roccat Kone EMP
Software Windows 10 x64
Yup, I think all those cards I am looking for have SFP+ ports, and I'm waiting for a price drop on TP-Link T1700G-28TQ switch.
 
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Got everything delivered today, the only issue I have/had is that the cards are HP branded 560SFP+ (which technically are Intel X520-DA2) but the description and the picture on ebay said otherwise.
I'll contact the seller and warn him about false description.
Despite that, I installed the cards, connected the cable, installed the drivers, configured the network and everything is ok for now.
Still doing some adjustments of the settings to optimize them for tape backup and to get the required performance on this ancient server I have. The main issue is CPU performance (running 2x Xeon Nocona at 3GHz and 4GB of DDR ECC RAM).

File copy was around 300MB/s to RAM due to slow SCSI disks on a quick dirty run I tried.
I will try to get some iperf or similar benchmark going tomorrow or during the weekend to see the maximum performance I can achieve.
And of course some pictures :)
 

Toothless

Tech, Games, and TPU!
Supporter
Joined
Mar 26, 2014
Messages
6,286 (2.70/day)
Location
Washington, USA
System Name Veraltet
Processor 3700x
Motherboard AsRock Taichi Ultimate x470
Cooling NH-D15 Chromax
Memory G.Skill 2x16GB 3200mhz
Video Card(s) MSI GTX1080 Ti Gaming X
Storage 2 NVMEs + 5 HDDs
Display(s) LG 34UB88-P + 2x AOC 2425W
Case Cosmos 1000
Audio Device(s) SteelSeries Arctis 5
Power Supply Rosewill Fortress 750w
Mouse Logitech G602
Keyboard G.Skill KM780 RGB (Brown switches)
Software Windows 10 Professional
Benchmark Scores Brrrrrrr
Jesus those chips are old. Might be worth considering a cheap upgrade to the 26xx series of Xeons for the storage speed bump.
 

Solaris17

Dainty Moderator
Staff member
Joined
Aug 16, 2005
Messages
21,689 (3.97/day)
Location
Florida
System Name Venslar
Processor I9 7980XE
Motherboard MSI x299 Tomahawk Arctic
Cooling EK Custom
Memory 32GB Corsair DDR4 3000mhz
Video Card(s) Nvidia Titan RTX
Storage 1x 250GB 960 EVO | 1x 500gb Intel 720p | 32TB SAN
Display(s) 3x AOC Q2577PWQ (2k IPS)
Case Inwin 303 White (Thermaltake Ring 120mm Purple accent)
Audio Device(s) Schiit Fulla 3 on Beyerdynamic DT 990 Pros
Power Supply Seasonic 1050W Snow
Mouse Roccat Kone Aimo White
Keyboard Ducky Shine 6 Snow White
Software Windows 10 x64 Pro
Not really, no real Gaines by upgrading the CPU at all. Iperf will show for sure but don’t forget you might need to cool the NICs
Oof mistook the arch those are brutally old. Do you maintain them for the sake of the tape drives? I’d be interested in iperf anyway
 
Last edited:
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Yes the CPU is very old, it's around P4 era but they are 64-bit.



The reason I didn't replace it yet because it just works - no extra funding is needed to keep it running. If I decide to get something used, but more modern I will definitely have to invest into it (the servvers I have I got for free and fully functional).
I also have a DL380G5 and a DL380G6 which I have a plan to use them as a replacement of this ancient ML. The only thing I need to design is an extra case with a fan to put the tape drives in to provide cooling, since I cannot fit them in a 2U chassis.

Ok so I did a quick and dirty run of iperf (didn't really do any optimization for it, just left the settings as I had for tape backup).


Yes the cards are not used to their full potential but when eventually I replace this server I will be able to use these same cards. For now they are doing their job just fine.
 

Solaris17

Dainty Moderator
Staff member
Joined
Aug 16, 2005
Messages
21,689 (3.97/day)
Location
Florida
System Name Venslar
Processor I9 7980XE
Motherboard MSI x299 Tomahawk Arctic
Cooling EK Custom
Memory 32GB Corsair DDR4 3000mhz
Video Card(s) Nvidia Titan RTX
Storage 1x 250GB 960 EVO | 1x 500gb Intel 720p | 32TB SAN
Display(s) 3x AOC Q2577PWQ (2k IPS)
Case Inwin 303 White (Thermaltake Ring 120mm Purple accent)
Audio Device(s) Schiit Fulla 3 on Beyerdynamic DT 990 Pros
Power Supply Seasonic 1050W Snow
Mouse Roccat Kone Aimo White
Keyboard Ducky Shine 6 Snow White
Software Windows 10 x64 Pro
Is this machine to machine though?

not machine to storage?

well I guess you aren’t doing it to an appliance so answered my own question
 
Joined
Apr 7, 2011
Messages
1,283 (0.38/day)
System Name Desktop
Processor Intel i7-3930k @ 4GHz Undervolted
Motherboard ASUS Sabertooth X79
Cooling Intel AIO
Memory 8x4GB DDR3 1866MHz
Video Card(s) EVGA GTX 970 SC
Storage LSI 9266-8i|3xHitachi SAS 15K600 450GB RAID5|6x WD RE Enterprise 4TB in RAID 10|HP Ultrium 1760 Tape
Display(s) HP ZR24w
Case Fractal Define XL Black
Audio Device(s) Schiit Modi Uber/Sony CDP-XA20ES/Pioneer CT-656>Sony TA-F630ESD>Sennheiser HD600
Power Supply Corsair HX850
Mouse Logitech G603
Keyboard Logitech G613
Software Windows 10 Pro x64
Machine to machine yes, for tape the maximum I need is around 120MB/s when it's at 2:1 compression, otherwise it's at 80MB/s.
Of course if I change to a newer LTO tape drive I will have to change the server also due to much higher speed demand. For LTO-8 you need around 900MB/s sustained when at 2:1 compression if you don't want any slow downs.
 
Top