How on the X99 platform I got 16X/16X/4X + 4X M.2 PCI-E operation

[davidm71]

As some of us X99 motherboard users know its only possible even with a 40 lane cpu to have 16X/16X/0 + 4X M.2 or 16X/16X/4X + M.2 PCH 2X operation. Just wanted to share how on my MSI Godlike X99 I was able to pull off 16X/16X/4X + 4X M.2 PCI-E mode. Was necessary to mod the bios to turn on Bifurcation mode of the last 8X lane pairs coming from the cpu. I had studied how on my motherboard MSI utilized switches to transfer the first half of the 8X PCI-E slot #5 to the M.2 onboard port. These switches were the same kind of switch used to transfer the other half of a 16X lane to the other 8X PCI-E Slot when more than two GPU's are being used. So it occurred to me that on the last PCI-E #5 slot the posterior half of that slot may still be active. So I set out to mod the bios to turn on bifurcation and purchased a Supermicro RSC-R2UU-2E4R 2U riser card and a 9cm riser extension cable. Hooked it up and right off the bat the OS initialized both drives at 4X PCI-E operation mode. Wrote a tutorial here.

 

[Flaky]

Great to see that working!

Would love to see similar thing done on 115x platform with cpu's lanes.
That would require either socket modding, or cpu modding/soldering, because the bifurcation is controlled by hardware signals send to cpu pins/pads.

That might result in sick ITX configurations, like real SLI, or dGPU and two NVMe drives with no DMI bottleneck involved.

Supermicro provides probably almost every variant of riser/splitter, even ones with embedded PLX chip.
One has to be careful though, because some of them are non-standard, and dedicated to their motherboards only.

 

[davidm71]

Well not sure about 115x. I looked into my Z270M mAtx bios and could not figure it out. Think the reason the Supermicro board worked out is because X99 is essentially a server based platform and the Supermicro tech support warned me it may not have worked but I am glad it did! Makes me wonder about if it be possible to run a raid off the slot while you still use the M.2 off of the PCH.

 

[uuuaaaaaa]

Proper work! Saluted!

 

[Sasqui]

Bifurcation... my new favorite word

 

[Flaky]

Well not sure about 115x. I looked into my Z270M mAtx bios and could not figure it out.

Because bifurcation on mainstream platforms is controlled in hardware, check CFG pin assignment -> 7th Gen Platform Spec (2.2.1)

Are you up for next challenge?

 

[davidm71]

Well as interesting as a challenge that is think i need to take a break. For one I don’t understand what processor signals are or even how to modify them. Table 6-7 in the datasheet details the signal configuration needed to set it at 8x 8x from 16x but I don’t understand if this is a binary bit needing to be set or what?

Thanks

 

[Toothless]

I gotta ask which case is that to have your drive sit upright?

 

[davidm71]

Coolermaster Cosmos 2

 

[davidm71]

Having a little issue with my Plextor drive forgetting to set its self at pci-e gen 3 speeds after the computer has been shut down (warm - still powered). HWInfo will say its at a 2.5 GT/S link speed though still 4X lanes. The only fix I have found is to shut off the bifurcation mod and turn it on again, or uninstall the driver and reinstall the driver. Think it has to do with L0s + L1 Latency values. Also think its fixable through software though not sure how. Like the Plextor goes to sleep and not wake up out of a low power mode.

Thanks

 

[erixx]

Check if there are reports of the Plextor doing it with all setups...

 

[davidm71]

You mean in other machines? Thinking about undoing the bifurcation and see what happens.

 

[er557]

That's pretty cool mod, nice. Thank god I dont have to deal with lanes ,as for two xeons= 80 lanes total, and each of the 7 pcie slots can be populated @x16 3.0.

 

[davidm71]

Aren't you lucky. I'm beginning to think this is a problem with the drive.

 

[davidm71]

Well I ran some more tests and it seems that a side effect of bifurcating the PCI-E 8X lane in two is that the Plextor occasionally sets itself to 2.5 GT/S mode. It doesn't happen when the bios hack is turned off and its not related to the riser cable. Interestingly enough the other NVME drive, the Samsung 950 Pro on the other end of the 4X switch, doesn't behave this way. So I think its something to do with the required latencies of the drive possibly the Payload side required. Not sure.

 

[TristanDuboisOLG]

Is it possible to get some documentation on how you did this? I'm part of the SFF community and not only are there a lot of people out there that could benefit from this, but I recently decided to make a SFF server and could really use it.

 

[Agent-Orange]

I would like to know, it that is also possible on an Asus Rampage 5 Extreme, to activate bifurcation on a modded bios.

 

[davidm71]

Wrote a guide some time back on how I did it.

https://www.win-raid.com/t3323f16-Guide-How-to-Bifurcate-a-PCI-E-slot.html#msg45527