On a roll: A screen tracks Nvidia as a trader works in the New York Stock Exchange. The company controls about 80% of high-end AI chip market, a position that has sent its stock market value up 40% so far this year. — Reuters
SAN FRANCISCO: Nvidia is building a new business unit focused on designing bespoke chips for cloud computing firms and others, including advanced artificial intelligence (AI) processors, nine sources familiar with its plans tells Reuters.
The dominant global designer and supplier of AI chips aims to capture a portion of an exploding market for custom AI chips and shield itself from the growing number of companies pursuing alternatives to its products.
The Santa Clara, California-based company controls about 80% of high-end AI chip market, a position that has sent its stock market value up 40% so far this year to US$1.73 trillion after it more than tripled in 2023.
Nvidia’s customers, which include ChatGPT creator OpenAI, Microsoft, Alphabet and Meta Platforms , have raced to snap up the dwindling supply of its chips to compete in the fast-emerging generative AI sector.
Its H100 and A100 chips serve as a generalised, all-purpose AI processor for many of those major customers. But the tech companies have started to develop their own internal chips for specific needs. Doing so helps reduce energy consumption, and potentially can shrink the cost and time to design.
Nvidia is now attempting to play a role in helping these companies develop custom AI chips that have flowed to rival firms such as Broadcom and Marvell Technology, said the sources, who declined to be identified because they were not authorised to speak publicly.
“If you’re really trying to optimise on things like power, or optimise on cost for your application, you can’t afford to go drop an H100 or A100 in there,” Greg Reichow, general partner at venture capital firm Eclipse Ventures said in an interview.
“You want to have the exact right mixture of compute and just the kind of compute that you need.”
Nvidia does not disclose H100 prices, which are higher than for the prior-generation A100, but each chip can sell for US$16,000 to US$100,000 depending on volume and other factors.
Meta plans to bring its total stock to 350,000 H100s this year.
Nvidia officials have met with representatives from Amazon.com, Meta, Microsoft, Google and OpenAI to discuss making custom chips for them, two sources familiar with the meetings said.
Beyond data centre chips, Nvidia has pursued telecom, automotive and video game customers.
Nvidia shares rose 2.75% after the Reuters report, helping lift chip stocks overall. Marvell shares dropped 2.78%.
In 2022, Nvidia said it would let third-party customers integrate some of its proprietary networking technology with their own chips.
It has said nothing about the programme since, and Reuters is reporting its wider ambitions for the first time.
A Nvidia spokesperson declined to comment beyond the company’s 2022 announcement.
Dina McKinney, a former Advanced Micro Devices and Marvell executive, heads Nvidia’s custom unit and her team’s goal is to make its technology available for customers in cloud, 5G wireless, video games and automotives, a LinkedIn profile said. Those mentions were scrubbed and her title changed after Reuters sought comment from Nvidia.
Amazon, Google, Microsoft, Meta and OpenAI declined to comment.
US$30bil market
According to estimates from research firm 650 Group’s Alan Weckel, the data centre custom chip market will grow to as much as US$10bil this year, and double that in 2025.
The broader custom chip market was worth roughly US$30bil in 2023, which amounts to roughly 5% of annual global chip sales, according to Needham analyst Charles Shi.
Currently, custom silicon design for data centres is dominated by Broadcom and Marvell.
In a typical arrangement, a design partner such as Nvidia would offer intellectual property and technology, but leave the chip fabrication, packaging and additional steps to Taiwan Semiconductor Manufacturing Co or another contract chip manufacturer.
Nvidia moving into this territory has the potential to eat into Broadcom and Marvell sales.
“With Broadcom’s custom silicon business touching US$10bil, and Marvell’s around US$2bil, this is a real threat,” said Dylan Patel, founder of silicon research group SemiAnalysis.
“It’s a real big negative – there’s more competition entering the fray.”
Beyond AI
Nvidia is in talks with telecom infrastructure builder Ericsson for a wireless chip that includes the chip designer’s graphics processing unit technology, two sources familiar with the discussions said. Ericsson declined to comment.
650 Group’s Weckle expects the telecom custom chip market to remain flat at roughly US$4bil to US$5bil a year.
Nvidia also plans to target the automotive and video game markets, according to sources and public social media postings.
Weckel expects the custom auto market to grow consistently from its current US$6bil to US$8bil range at 20% a year, and the US$7bil to US$8bil video game custom chip market could increase with the next-generation consoles from Xbox and Sony.
Nintendo’s current Switch handheld console already includes Nvidia’s Tegra X1 chip. A new version of the Switch console expected this year is likely to include a Nvidia custom design, one source said.
Nintendo declined to comment. — Reuters
