Thursday, February 29th 2024
Tiny Corp. Builds AI Platform with Six AMD Radeon RX 7900 XTX GPUs
Tiny Corp., a neural network framework specialist, has revealed intimate details about the ongoing development and building of its "tinybox" system: "I don't think there's much value in secrecy. We have the parts to build 12 boxes and a case that's pretty close to final. Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this. Our BOM cost is around $10k, and we are selling them for $15k. We've put a year of engineering into this, it's a lot harder than it first seemed. You are welcome to believe me or not, but unless you are building in huge quantity, you are getting a great deal for $15k." The startup has taken the unusual step of integrating Team Red's current flagship gaming GPU into its AI-crunching platform. Tiny Corp. founder—George Hotz—has documented his past rejections of NVIDIA AI hardware on social media, but TinyBox will not be running AMD's latest Instinct MI300X accelerators. RDNA 3.0 is seemingly favored over CDNA 3.0—perhaps due to growing industry demand for enterprise-grade GPUs.
The rack-mounted 12U TinyBox build houses an AMD EPYC 7532 processor with 128 GB of system memory. Five 1 TB SN850X SSDs take care of storage duties (4 in raid, 1 for boot), and an unoccupied 16x OCP 3.0 slot is designated for networking tasks Two 1600 W PSUs provide necessary electrical juice. The Tiny Corp. social media picture feed indicates that they have acquired a pile of XFX Speedster MERC310 RX 7900 XTX graphics cards—six units are hooked up inside of each TinyBox system. Hotz's young startup has ambitious plans: "The system image shipping with the box will be Ubuntu 22.04. It will only include tinygrad out of the box, but PyTorch and JAX support on AMD have come a long way, and your hardware is your hardware. We make money either way, you are welcome to buy it for any purpose. The goal of the tiny corp is to commoditize the petaflop, and we believe tinygrad is the best way to do it. Solving problems in software is cheaper than in hardware. tinygrad will elucidate the deep structure of what neural networks are. We have 583 preorders, and next week we'll place an order for 100 sets of parts. This is $1M in outlay. We will also ship five of the 12 boxes we have to a few early people who I've communicated with. For everyone else, they start shipping in April. The production line started running yesterday."
Sources:
TinyGrad.org, Tom's Hardware, Wccftech, Tinygrad Tweet
The rack-mounted 12U TinyBox build houses an AMD EPYC 7532 processor with 128 GB of system memory. Five 1 TB SN850X SSDs take care of storage duties (4 in raid, 1 for boot), and an unoccupied 16x OCP 3.0 slot is designated for networking tasks Two 1600 W PSUs provide necessary electrical juice. The Tiny Corp. social media picture feed indicates that they have acquired a pile of XFX Speedster MERC310 RX 7900 XTX graphics cards—six units are hooked up inside of each TinyBox system. Hotz's young startup has ambitious plans: "The system image shipping with the box will be Ubuntu 22.04. It will only include tinygrad out of the box, but PyTorch and JAX support on AMD have come a long way, and your hardware is your hardware. We make money either way, you are welcome to buy it for any purpose. The goal of the tiny corp is to commoditize the petaflop, and we believe tinygrad is the best way to do it. Solving problems in software is cheaper than in hardware. tinygrad will elucidate the deep structure of what neural networks are. We have 583 preorders, and next week we'll place an order for 100 sets of parts. This is $1M in outlay. We will also ship five of the 12 boxes we have to a few early people who I've communicated with. For everyone else, they start shipping in April. The production line started running yesterday."
25 Comments on Tiny Corp. Builds AI Platform with Six AMD Radeon RX 7900 XTX GPUs
good news for AMD I guess.
my guess is you can probably double that number within a few months as word spreads
The natural trade-off is the larger footprint and power consumption this whole thing takes up, but considering that all of the components are more readily available and easy to upgrade down the line and can run different AI code, it's a decent deal compared to similarly performing AI hardware.
I do sometimes wonder how powerful we could be as a species if competition didn't exist and we simply strive for greatness for greatness sake.
To the point that only a few can purchase bleeding edge tech while the rest of us go with older equipment and/or lesser upgrade cycles.
That's smarter than half the PhD students I have come across.
just link up a bunch of gpus in a rack and buuurn that shit
Inb4 the next bubble. AI hasnt done F all yet for us. And it likely wont
Its burning energy for little production just like a lot of crypto.
Its driving a bubble of hardware that will be obsolete in 2 years.
Its driving up the cost for gamers as it takes up fab, assembly, memory, IC and other components.
So much in so little time.
Funny how that coincides with a lacking perspective on our future outlook wrt the planet and climate. Charge forward into stupidity, ignorance is bliss. One fad after another.
Disgusting.