Friday, February 23rd 2024
NVIDIA Prepared to Offer Custom Chip Designs to AI Clients
NVIDIA is reported to be setting up an AI-focused semi-custom chip design business unit, according to inside sources known to Reuters—it is believed that Team Green leadership is adapting to demands leveraged by key data-center customers. Many companies are seeking cheaper alternatives, or have devised their own designs (budget/war chest permitting)—NVIDIA's current range of AI GPUs are simply off-the-shelf solutions. OpenAI has generated the most industry noise—their alleged early 2024 fund-raising pursuits have attracted plenty of speculative/kind-of-serious interest from notable semiconductor personalities.
Team Green is seemingly reacting to emerging market trends—Jensen Huang (CEO, president and co-founder) has hinted that NVIDIA custom chip designing services are on the cusp. Stephen Nellis—a Reuters reporter specializing in tech industry developments—has highlighted select NVIDIA boss quotes from an incoming interview piece: "We're always open to do that. Usually, the customization, after some discussion, could fall into system reconfigurations or recompositions of systems." The Team Green chief teased that his engineering team is prepared to take on the challenge meeting exact requests: "But if it's not possible to do that, we're more than happy to do a custom chip. And the benefit to the customer, as you can imagine, is really quite terrific. It allows them to extend our architecture with their know-how and their proprietary information." The rumored NVIDIA semi-custom chip design business unit could be introduced in an official capacity at next month's GTC 2024 Conference.
Sources:
Stephen Nellis Tweet, Wccftech, Business Insider
Team Green is seemingly reacting to emerging market trends—Jensen Huang (CEO, president and co-founder) has hinted that NVIDIA custom chip designing services are on the cusp. Stephen Nellis—a Reuters reporter specializing in tech industry developments—has highlighted select NVIDIA boss quotes from an incoming interview piece: "We're always open to do that. Usually, the customization, after some discussion, could fall into system reconfigurations or recompositions of systems." The Team Green chief teased that his engineering team is prepared to take on the challenge meeting exact requests: "But if it's not possible to do that, we're more than happy to do a custom chip. And the benefit to the customer, as you can imagine, is really quite terrific. It allows them to extend our architecture with their know-how and their proprietary information." The rumored NVIDIA semi-custom chip design business unit could be introduced in an official capacity at next month's GTC 2024 Conference.
30 Comments on NVIDIA Prepared to Offer Custom Chip Designs to AI Clients
Anyhow this makes sense like a bakery branching out into custom cake orders.
The big question is when will AMD and Intel offer similar services to compete with this rumored Nvidia offering? Nvidia is already dominating marketshare right now. Staying with off-the-shelf designs probably isn't going to be competitive advantage.
They did this over time. Modifying Arm cores, eventually moving parts to their own design. Hell, Apple used Imagination PowerVR graphics cores while they had moved the CPU cores to their own design. Apple chips with their first ML cores came out about 5 years ago within a year of Nvidia's Tensor cores.
And Apple had a really great VLSI team because they were designing their own ASICs. You don't just wake up one day and say "let's make an AI chip" and see it tape out six months from now. Your snot-nosed 22-year old new hires aren't going to be rolling out AI ASICs anytime soon.
For now a lot of companies want an advantage while they develop their own in-house solutions. That's what Nvidia is addressing. And if Big Company A launches their new AI chip in 2030, what happens in 2025, 2026, 2027, 2028, 2029? And there will be new small companies that simply don't have the resources to launch into a multi-year chip development program with no guarantee that they will stay afloat.
Nvidia is doing custom chips a little time now, to circumvent restrictions and keep selling to China.
People designing their own silicon will have to find their own software solutions which also takes a lot of time, money and expertise. Just putting a bunch of transistors on a wafer is not the sole task at hand.
I know a lot of TPU commenters tend to ignore the software development aspects. It's not just a spec sheet or benchmark.
You can't just click your fingers and scale it up.
It can take even longer than that depending on if you need to build power infrastructure as well. Which is probable.
No need for Nvidia and paying 80% nosebleed margins on their chips. So obviously they'll try to retain that business.
This was discussed heavily when the ARC graphics cards debuted and I said several times that the silicon looked fine, the drivers needed work. Yet most people online fixated on core counts, transistors, clock speeds, etc. rather than take software into account.
Intel has a better software engineering team than AMD. They will have a better chance at taking a dent out of Nvidia's AI accelerator marketshare in the next few years.
AMD really needs to step up their software for any chance of staying relevant.
Look at Apple's acquisition of P.A. Semi. It took nearly ten years to get a shipping product from the result of this acquisition.
I worked at a place that designed semiconductors. You don't just wave a magic wand and have a design that tapes out in 1.5 years. That wasn't possible thirty years ago and it certainly isn't possible today.
That's why Nvidia is doing all of this. They know some of these potential customers may roll their own in a few years. But not tomorrow, not next month, not next year.
Let's say Nvidia signs up 50 customers for this new custom chip design business. I doubt if two of them will tape out their own chips in 1.5 years. And whatever does tape out may not be competitive enough to whatever Nvidia has in 2026.
Again, all of this chip design work will be accelerated by AI assistance. It takes computer chips to design computer chips in 2024. You still need someplace to start. The era of using a pencil, pad of paper and sliderule to design silicon is over.
Look at AMD. They've been in the GPU business for decades and they still can't take a major bite out of Nvidia. It takes enormous skill and resources to play in the semiconductor business today.
The ones that do will not sign up for this. But that's just a handful of companies on the planet.
I live in a place where autonomous vehicles are heavily tested on public roads. Pretty much every single one of them has an Nvidia GPU onboard. This includes some of the largest car corporations on the planet. That's right, Toyota, Ford, GM, they all use Nvidia GPUs for this usage case because they don't have in-house silicon good enough to do the job.
And as I pointed out earlier, it's not just the silicon. There's the software side of things.
You guys seriously think that Nvidia will be offering something that no one wants? Doesn't it make sense that maybe Nvidia got enough feedback (privately) where they thought they could turning this into a business? Isn't that how a lot of businesses start?
Nvidia is the noob here when it comes to designing custom chips.
I was driving up 101 a few days ago and I didn't see any billboards proclaiming: "Custom AI silicon! Get a tailored design taped out in 18 months! Call now for a free quote!"I know AMD does custom SoC design work beyond Sony and Microsoft. I'm an indirect shareholder of AMD, Intel, and Nvidia, I read the occasional annual report.
But for sure, AMD's custom SoC design work has not materialized into major revenue.
Again, people here are ignoring the software side of things. In order to use AI chips you need software. Currently, Nvidia's AI software development environment greatly exceeds that of AMD, Intel, and others in terms of addressing a broad selection of interests.
Lots of companies will eventually roll their own but NOT TOMORROW, NOT NEXT MONTH, NOT NEXT YEAR.
If there are comparable products and services being offered in the custom AI chip design marketplace, who are those competitors? List six or seven companies. List their customers. And their quarterly revenues over the past 24-36 months. Don't worry, I can look up AMD, NVDA, and INTC quarterly reports online.
And let's not forget that there's a manufacturing component. Let's just say that magically I have an ASIC design ready to tape out, gifted by my fairy chip godmother. What chance do I have calling up TSMC and asking them to set aside some wafers on their most advanced node? Credit check? Oh yes, of course, here's my driver's license.
Nvidia already has that leverage. How much business has Nvidia been giving to TSMC and Samsung over the past ten years?
Let's skip past Berkshire-Hathaway and look at the next few companies: Visa, JPMorganChase, UnitedHealth, Walmart, and Mastercard. Do you think any of these companies have a VLSI team that is ready to tape out a competitive AI accelator design in 1.5 years? Or Exxon Mobil, Johnson & Johnson, Proctor & Gamble, Home Depot, Merck?
Even Tesla has not weaned themselves off of Nvidia, AMD, and Intel silicon.
Let's face it, Nvidia doesn't have the resources to accept everyone knocking at their door. There will be other companies who will offer AI chip design services. But who are they?
I bet the top 20% of the S&P 500 makes about 85% of the revenue.
If Nvidia can handle something like 50 custom chip customers, they can pretty much focus on the S&P 100. What is going to happen is a temporary squeeze on engineering talent. These companies will be fighting to attract the right talent.
In the same way, if you asked ten people to start bakeries, not all of them would open on the same day with the same volume, same foot traffic, same revenue, etc.
Again, please list the companies that can go from concept to tapeout in 1.5 years. I would love to hear about these players. You're the one who brought it up.
www.techpowerup.com/318351/amd-instinct-mi300x-released-at-opportune-moment-nvidia-ai-gpus-in-short-supply
www.techpowerup.com/319307/intel-and-ohio-supercomputer-center-double-ai-processing-power-with-new-hpc-cluster
www.techpowerup.com/319397/arm-launches-next-generation-neoverse-css-v3-and-n3-designs-for-cloud-hpc-and-ai-acceleration
www.techpowerup.com/319320/intel-foundry-services-ifs-and-cadence-design-systems-expand-partnership-on-soc-design
github.com/ray-project/llmperf-leaderboard
From the last few days of news. Nvidia's lead is due to them selling AI equipment masked as something else, before anyone else. That lead will not last because they have general designs that are sure to be beat by custom SoC's. That's why Jensen is trying to entice them under his jacket.
Your inclusion of the Arm Neoverse article is amusing since it quotes how Arm CPUs work in conjunction with Nvidia Grace and Nvidia Bluefield. Moreover, selling chip designs is Arm's business.
We already know that AMD and Intel have AI products, there's nothing that shows that these products are the result of some sort of custom chip design business. The AMD MI300 Instinct product appears to be off the shelf, just like Nvidia's current offerings.
Just tossing out random articles of other companies' off-the-shelf AI products does not negate the possible desirability for custom chip design services.
To each there own bro, you can go on beliving it takes a decade to make an ASIC. I just thought it would be friently to let you know it doesnt.
I will also throw in as a bonus just for you, that if you think any of these companies arent working every microsecond towards plaform diversification even if news outlets mention they are buying "X GPUs" from %manufacturer% you are being decived.
Nvidia has a market for custom order chips no doubt buddy, if you dont want to belive the rest of it...well I dont really care. :)
Maybe you can release after five years with a less capable product. Or something that has no cost savings. Or something with a truncated feature list.
Remember that all of these companies (Intel, AMD, Nvidia, and others) have released products that were flops despite months (sometimes years) of work. Just because it tapes out doesn't mean it's good.
Ah, so you say you know companies that can tape out in 1.5 years but rather than point them out without comment, you leave an even blurrier and unverifiable story and bow out of coughing up names. That's very convenient. And if they are known for and took pride in having such quick turnaround, this wouldn't really be hiding anything, would it?Oh, I believe that any company that signs up for Nvidia's rumored custom chip service will be looking into doing it themselves.
But that said, do you bake your own bread? Pasta? Cookies? Beer?
Look at your jeans. The fabric likely came from a loom run by another company. From cotton grown by farmers elsewhere.
Someone gives you a manual pasta machine, an egg, a cup of flour. Do you think you can pump out pasta as good as the boxed stuff from the supermarket? Or the pricier fresh stuff in the deli department's refrigerator? The first time? The twentieth time? How much time would it take? Can do this for a restaurant that serves 150 covers at dinner?
Let's say Home Depot has some programmers running AI models on inventory management on Nvidia AI accelerators. Do you think they'll be ready to tape out a competitive chip design in 1.5 years?
Big green also makes more than just GPUs. Costom chip can mean a lot of things. Doesn't need to mean a GPU. In some spaces custom storage or network type controllers are worth a lot more than a custom AI or GPU though these are often also used.
It seems like a lot of people here think chip + GPU or x86 CPU and just ignore how small of the market that is.