Anyone have experience with infiniband?

[Rehmanpa]

Trying to figure out how to set up a home network using infiniband. I'm wondering if anyone here has had experience with it and could help me out. I'd like to buy some cheap, 1m infiniband cables, buy a bunch of the cable part, and then crimp the ends onto them so I can reduce the $$$ of the cabling.

Anyone know of I can do that, or have any recommendations on this stuff?

Thanks

 

[R-T-B]

Infiniband is generally used for longer distance interprocess communication on supercomputers.

I hope you brought your wallet... Nothing about this will be cheap.

 

[jboydgolfer]

Trying to figure out how to set up a home network using infiniband.

How large is your home, & are u certain ethernet wont suffice? Unless im not understanding, anything defined as "home network" is fine with gbethernet....atleast mostly

Also, im not sure if I understand your question.....r u asking if u can buy 1meter cables & join several together instead of buying a longer more $ one?? If so, more connections is never good.

 

[R-T-B]

How large is your home, & are u certain ethernet wont suffice?

Also, im not sure if I understand your question.....r u asking if u can buy 1meter cables & join several together instead of buying a longer more $ one?? If so, more connections is never good

Plus most infiniband cables are fiber. Passive copper can't go more than 10m

 

[Rehmanpa]

Infiniband is generally used for longer distance interprocess communication on supercomputers.

I hope you brought your wallet... Nothing about this will be cheap.

Infiniband is cheap as hell, at least compared to 10gb ethernet and fiber. Also, I can actually buy pcie cards that support more than 10gb speeds (a dual 40gb infiniband pcie adapter is about $40-$60 on ebay/amazon).

How large is your home, & are u certain ethernet wont suffice? Unless im not understanding, anything defined as "home network" is fine with gbethernet....atleast mostly

Also, im not sure if I understand your question.....r u asking if u can buy 1meter cables & join several together instead of buying a longer more $ one?? If so, more connections is never good.

I currently have gbethernet, and it's too slow for what I want to do. I want to build a nas box and have maximum speed for everything, and if I want to actively video edit/render off of it, I'll need to have more speed. I was looking at 10gb ethernet, but this is significantly cheaper, and I believe if I had the adapter which have two cables running out of one and create a tolken ring, I can effectively bypass the need of a switch (idk how much switches cost off the top of my head).


My biggest questions had been this: Could I buy a 1m infiniband cable, cut the ends off of it, and put those ends on a 15m long fiber cable so I don't have to buy the 15m cable with ends already on them to help shave off my costs?? If so, does anyone know where I could find a guide/wire outline for doing that? I've never worked with fiber and while I've heard it can be a pain, if I could get 40gbps running around my houses at theses kind of prices it would undoubtedly be worth it (at least for me).

 

[Rehmanpa]

Bump

 

[R-T-B]

Infiniband is cheap as hell, at least compared to 10gb ethernet and fiber. Also, I can actually buy pcie cards that support more than 10gb speeds (a dual 40gb infiniband pcie adapter is about $40-$60 on ebay/amazon).

Wikipedia didn't make it sound that way nor did my textbook way back when.

I think you must be missing something, frankly.

 

[R00kie]

The ends on the cables are fused, so no, but there are modules that you can use, and stick whatever cables you want, but they're 100 or more bucks each. So what you're asking is not cheap in any stretch of imagination.

 

[Rehmanpa]

Wikipedia didn't make it sound that way nor did my textbook way back when.

I think you must be missing something, frankly.

I just pulled these off of ebay in like a minute or two so it's not very exhaustive in terms of finding the cheapest prices.
https://www.ebay.com/itm/592520-B21...115171&hash=item3aea85f036:g:vrMAAOSwAHBZ-zyS
https://www.ebay.com/itm/Mellanox-I...599444&hash=item2a899c79b7:g:Xb8AAOSw68NZgM8u

A 40Gb/s pcie cardfor only $28. It's MUCH cheaper and faster than 10gb ethernet, and significantly faster than my gigabit ethernet. The switch I found was only $220 and it has 8 40Gb/s ports on the thing. My questions mostly are portraying to the cable parts of things, as that's the last roadblock to going this route since it is pretty damn cheap (in my opinion at least compared to other options).

The ends on the cables are fused, so no, but there are modules that you can use, and stick whatever cables you want, but they're 100 or more bucks each. So what you're asking is not cheap in any stretch of imagination.

How are the ends fused together? Is there a kind of tool that I can use to "unfuse" them, or am I just stuck buying a $150 cable for 15m? I don't have any experience with stuff like this or with Fiber in general, I just know that with ethernet I can crimp the ends on for whatever length I want. You're the first one to actually discuss the physical cable itself which is where I'm currently a bit stuck on, trying to figure out what kind of ends I'd need and how to make it work correctly.

 

[R00kie]

I just know that with ethernet I can crimp the ends on for whatever length I want. You're the first one to actually discuss the physical cable itself which is where I'm currently a bit stuck on, trying to figure out what kind of ends I'd need and how to make it work correctly.

Its just not that simple, you cant just crimp a fiber cable:

There are QSFP modules that you can buy, but as youll probably see from the link below, you wont be able to do that cheaply:
https://www.ebay.com/sch/i.html?_fr...Xqsfp+m.TRS0&_nkw=qsfp+module&redirect=mobile

 

[Rehmanpa]

Its just not that simple, you cant just crimp a fiber cable:

There are QSFP modules that you can buy, but as youll probably see from the link below, you wont be able to do that cheaply:
https://www.ebay.com/sch/i.html?_fr...Xqsfp+m.TRS0&_nkw=qsfp+module&redirect=mobile

That's really good to know. The metal part you have in your picture, that is the part that attaches to the cable, right? Can you take that end off and put it onto a new cable? If so, could I get a 3ft cable with those already attached and then take those off and reattach those to a longer cable to save off some $$?

 

[xorbe]

I thought infiniband was for high-speed cpu interconnect between local boxes. Does it act like a standard network interface with drivers or something?

 

[AhokZYashA]

im pretty sure infiniband cables are pre-made in factory,
dunno if you can open it up and re-crimp it to the length you want,

i think you'd better using teamed 10GbE for 20Gb of bandwidth

because infiniband is not something like ethernet, its a completely different protocol altogether

 

[Rehmanpa]

im pretty sure infiniband cables are pre-made in factory,
dunno if you can open it up and re-crimp it to the length you want,

i think you'd better using teamed 10GbE for 20Gb of bandwidth

because infiniband is not something like ethernet, its a completely different protocol altogether

Dude, it is basically really fast Ethernet. It works just fine for networking. I just want to know about the cable ends, not infiniband itself.

10gb Ethernet is significantly more money and it's slower. The only thing I'm asking about is if anyone has messed around with the cable ends or not. I know I'm going to go infiniband it just is a matter of how soon depending on cable costs.

 

[Frick]

Here's a similar thread worth reading.

https://forums.servethehome.com/index.php?threads/infiniband-setup-advice-needed.8094/

As one user puts is:

"In terms of rolling your own fibre/re-splicing those cut ends, I really wouldn't bother. Cutting, polishing and terminating fibre to a sufficiently high standard is quite the skill and the kit to do it, and more importantly test it is very expensive."

There are ways though, of course there are. I don't know what toolkits to buy, but I am assuming they are not cheap and I am assuming it will be a skill involved. Fiber never is easy, you can't just shove them onto a circuit board as you can with copper. Another question is how much money and time are you willing to spend on basically learning a skill you likely never need again and don't have to learn in the first place?

 

[Rehmanpa]

Here's a similar thread worth reading.

https://forums.servethehome.com/index.php?threads/infiniband-setup-advice-needed.8094/

As one user puts is:

"In terms of rolling your own fibre/re-splicing those cut ends, I really wouldn't bother. Cutting, polishing and terminating fibre to a sufficiently high standard is quite the skill and the kit to do it, and more importantly test it is very expensive."

There are ways though, of course there are. I don't know what toolkits to buy, but I am assuming they are not cheap and I am assuming it will be a skill involved. Fiber never is easy, you can't just shove them onto a circuit board as you can with copper. Another question is how much money and time are you willing to spend on basically learning a skill you likely never need again and don't have to learn in the first place?

Thank you for your reply, a lot of useful information. It's seeming like I'm better off either buying the cables already to go, or I'd have to buy the right ends and go through a big hassle. If I'm going to be spending a bit of $$$ on this i may as well do it right. I'll read through that forum later, but from what I glanced it looked really good.

 

[Frick]

Please come again if you reach the conclusion you'll do it; I too have been interested in running Infinoband at home for a while now. Theoretically anyway. A few years back it definitely wasn't worth it for any home application, but things change.

 

[Rehmanpa]

Please come again if you reach the conclusion you'll do it; I too have been interested in running Infinoband at home for a while now. Theoretically anyway. A few years back it definitely wasn't worth it for any home application, but things change.

Yeah and it's the cheapest option for this kind of speed. People here don't seem to get that...

 

[R-T-B]

Yeah and it's the cheapest option for this kind of speed. People here don't seem to get that...

I'll fully admit I only know what my textbook fed me.

If you can prove via documentation that it's a cheaper option (including labor) than say teamed 10gbps ethernet, I'm sure several here would be interested.