Tiny Corp. Builds AI Platform with Six AMD Radeon RX 7900 XTX GPUs

[T0@st]

Tiny Corp., a neural network framework specialist, has revealed intimate details about the ongoing development and building of its "tinybox" system: "I don't think there's much value in secrecy. We have the parts to build 12 boxes and a case that's pretty close to final. Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this. Our BOM cost is around $10k, and we are selling them for $15k. We've put a year of engineering into this, it's a lot harder than it first seemed. You are welcome to believe me or not, but unless you are building in huge quantity, you are getting a great deal for $15k." The startup has taken the unusual step of integrating Team Red's current flagship gaming GPU into its AI-crunching platform. Tiny Corp. founder—George Hotz—has documented his past rejections of NVIDIA AI hardware on social media, but TinyBox will not be running AMD's latest Instinct MI300X accelerators. RDNA 3.0 is seemingly favored over CDNA 3.0—perhaps due to growing industry demand for enterprise-grade GPUs.

The rack-mounted 12U TinyBox build houses an AMD EPYC 7532 processor with 128 GB of system memory. Five 1 TB SN850X SSDs take care of storage duties (4 in raid, 1 for boot), and an unoccupied 16x OCP 3.0 slot is designated for networking tasks Two 1600 W PSUs provide necessary electrical juice. The Tiny Corp. social media picture feed indicates that they have acquired a pile of XFX Speedster MERC310 RX 7900 XTX graphics cards—six units are hooked up inside of each TinyBox system. Hotz's young startup has ambitious plans: "The system image shipping with the box will be Ubuntu 22.04. It will only include tinygrad out of the box, but PyTorch and JAX support on AMD have come a long way, and your hardware is your hardware. We make money either way, you are welcome to buy it for any purpose. The goal of the tiny corp is to commoditize the petaflop, and we believe tinygrad is the best way to do it. Solving problems in software is cheaper than in hardware. tinygrad will elucidate the deep structure of what neural networks are. We have 583 preorders, and next week we'll place an order for 100 sets of parts. This is $1M in outlay. We will also ship five of the 12 boxes we have to a few early people who I've communicated with. For everyone else, they start shipping in April. The production line started running yesterday."



View at TechPowerUp Main Site | Source

 

[FreedomEclipse]

Thats a rather tiny amount of 7900XT's

 

[Space Lynx]

So what your saying is, I shouldn't have sold my computer the other day. Cause gpu shortages incoming, lmao

 

[john_]

"We have 583 preorders"

If a .....tiny corporation, using AMD GPUs instead of Nvidia GPUs gets 583 preorders, that seems like a very clear indication of the AI craze we hear from about everyone. I mean, we hear it, but reading about 583 preorders for AMD GPUs and even more, gaming GPUs, ....damn...... this looks like crypto mining all over again.

 

[Space Lynx]

If a .....tiny corporation, using AMD GPUs instead of Nvidia GPUs gets 583 preorders, that seems like a very clear indication of the AI craze we hear from about everyone. I mean, we hear it, but reading about 583 preorders for AMD GPUs and even more, gaming GPUs, ....damn...... this looks like crypto mining all over again.

6 gpus per build, 583 orders, thats 3500 7900 xt's

good news for AMD I guess.

my guess is you can probably double that number within a few months as word spreads

 

[Denver]

A single mi300x beats 10 XTXs put together. :')

 

[jesdals]

1, 2, 3 and GPU shortage is back OMG hope this dosent catch on

 

[Space Lynx]

A single mi300x beats 10 XTXs put together. :')

yeah but production is short on those. and nvidia has bought all the fab times... that's the whole point here, people wanting AI and not having to wait around.

1, 2, 3 and GPU shortage is back OMG hope this dosent catch on

it probably will, cause humans are a failed species in many ways unfortunately.

 

[FeelinFroggy]

Looks like crypto mining all over again.

 

[AnarchoPrimitiv]

Darn, now I feel like I have to by a new videocard now and I wanted to wait

 

[TechLurker]

yeah but production is short on those. and nvidia has bought all the fab times... that's the whole point here, people wanting AI and not having to wait around.

That and using cheaper off-the-shelf solutions instead of trying to invest in specialty hardware and charging at a premium to recoup the development and investment costs. That's part of their mission statement.

The natural trade-off is the larger footprint and power consumption this whole thing takes up, but considering that all of the components are more readily available and easy to upgrade down the line and can run different AI code, it's a decent deal compared to similarly performing AI hardware.

 

[Space Lynx]

As long as the AI is used for good stuff I have no problem with it really. If I were a small startup company making new medicines (I have seen a few around) I would be looking at adding a small AI system to my inventory and have it program dedicated for medical stuff. Better yet, imagine if all the major companies stopped competing, and made one giant fucking AI, imagine the things it could solve... air turbulence, new designed AI plane makes it vanish overnight, cancer? gone, most efficient and optimal ways to improve soil health? fixed. Improve solar panel absorption rate by mixing these three things together at this solution - boom done.

I do sometimes wonder how powerful we could be as a species if competition didn't exist and we simply strive for greatness for greatness sake.

 

[Icon Charlie]

If a .....tiny corporation, using AMD GPUs instead of Nvidia GPUs gets 583 preorders, that seems like a very clear indication of the AI craze we hear from about everyone. I mean, we hear it, but reading about 583 preorders for AMD GPUs and even more, gaming GPUs, ....damn...... this looks like crypto mining all over again.

I agree with you on this. From my perspective this is another bubble as has been over hyped to a point that this will directly/indirectly keep tech components pricing high.
To the point that only a few can purchase bleeding edge tech while the rest of us go with older equipment and/or lesser upgrade cycles.

 

[Space Lynx]

I agree with you on this. From my perspective this is another bubble as has been over hyped to a point that this will directly/indirectly keep tech components pricing high.
To the point that only a few can purchase bleeding edge tech while the rest of us go with older equipment and/or lesser upgrade cycles.

I don't think it is a gimmick, even asking CopIlot some very complex questions, I noticed it got something wrong, so I confronted it on this with a rebuttal, and it apologized, expanded its resources, agreed with me, explain what happened, and said it understands better now.

That's smarter than half the PhD students I have come across.

 

[mechtech]

A single mi300x beats 10 XTXs put together. :')

How much does 1 unit cost?

 

[Space Lynx]

How much does 1 unit cost?

a hell of a lot more than 10 XTX's. assuming you can even order one, as said before Nvidia bought all the fab times, so yeah. AMD vaporware most likely.

 

[Chaitanya]

it probably will, cause humans are a failed species in many ways unfortunately.

Agent Smith was right all along.

 

[Denver]

How much does 1 unit cost?

Supposedly U$ 15k, All current and future production must have already been sold. AMD is now struggling to scale up production due to the absence of a viable alternative to CoWoS.

 

[Space Lynx]

Agent Smith was right all along.

those movies were so ahead of their time. kids today are too busy scrolling tiktok, but us? we were contemplating philosophy that was deep as fk lol

 

[Wirko]

Supposedly U$ 15k, All current and future production must have already been sold.

Sure it can be sold for more than that, therefore it's sold for more than that. $15k is the speculated price for early supporters, Kickstarter style. Such as Microsoft. Those may even be willing to accept hardware with bugs, just to get it early.

AMD is now struggling to scale up production due to the absence of a viable alternative to CoWoS.

They are in a much better position financially than a couple years ago. If they are smart (which I believe), they financed and reserved a solid amount of CoWoS and other advanced packaging capacities in time. That apparently is a narrower bottleneck than wafer fabrication these days.

 

[T1beriu]

You have no idea

assuming you can even order one, as said before Nvidia bought all the fab times, so yeah. AMD vaporware most likely.

You have no clue what you're talking about. MI300 is AMD's most successful launch ever. Lisa Su announced a few weeks ago that AMD already sold more that $2 Billion of MI300 products and they have capacity to fulfill orders of $3.6-4 Billion in 2024.

 

[Vayra86]

If a .....tiny corporation, using AMD GPUs instead of Nvidia GPUs gets 583 preorders, that seems like a very clear indication of the AI craze we hear from about everyone. I mean, we hear it, but reading about 583 preorders for AMD GPUs and even more, gaming GPUs, ....damn...... this looks like crypto mining all over again.

Yep...

just link up a bunch of gpus in a rack and buuurn that shit

Inb4 the next bubble. AI hasnt done F all yet for us. And it likely wont

 

[Steevo]

Yep...

just link up a bunch of gpus in a rack and buuurn that shit

Inb4 the next bubble. AI hasnt done F all yet for us. And it likely wont

It has too, its created a lot of comedy, look at Gemini and its utter flubs. I tried Copilot until it got pissy, then it got hard uninstalled.

Its burning energy for little production just like a lot of crypto.

Its driving a bubble of hardware that will be obsolete in 2 years.

Its driving up the cost for gamers as it takes up fab, assembly, memory, IC and other components.

So much in so little time.

 

[Vayra86]

It has too, its created a lot of comedy, look at Gemini and its utter flubs. I tried Copilot until it got pissy, then it got hard uninstalled.

Its burning energy for little production just like a lot of crypto.

Its driving a bubble of hardware that will be obsolete in 2 years.

Its driving up the cost for gamers as it takes up fab, assembly, memory, IC and other components.

So much in so little time.

Fools and money - and we get lots of confirmations of fools all across the globe.

Funny how that coincides with a lacking perspective on our future outlook wrt the planet and climate. Charge forward into stupidity, ignorance is bliss. One fad after another.

Disgusting.

 

[Space Lynx]

Fools and money - and we get lots of confirmations of fools all across the globe.

Funny how that coincides with a lacking perspective on our future outlook wrt the planet and climate. Charge forward into stupidity, ignorance is bliss. One fad after another.

Disgusting.

and when all the resources start to run dry or the next famine comes, they will turn on each other like animals. humans were capable of so much, yet so little ~