Consulting giant Deloitte noted in November last year that inference workloads will be the next big thing in artificial intelligence (AI) in 2026. According to Deloitte, inference will account for two-thirds of AI computing power this year, up from 50% in 2025.
Deloitte estimates the market for inference-focused AI chips could reach $50 billion this year. McKinsey, on the other hand, estimates that AI inference workloads in data centers could jump from almost 21 gigawatts (GW) last year to 93 GW in 2030, clocking a compound annual growth rate (CAGR) of 35%.
Not surprisingly, there is a race among AI chipmakers to make inference-focused processors to capitalize on this lucrative growth opportunity. From Nvidia (NVDA 1.86%) to Advanced Micro Devices to Broadcom to Intel, everyone is trying to make the most efficient chips that can run AI inference applications cost-effectively in data centers and at the edge.
However, I believe that these semiconductor companies will be beaten by Arm Holdings (ARM +2.78%) in the inference era. Let's look at the reasons why.
Image source: The Motley Fool.
Arm Holdings is a top pick-and-shovel AI inference play
AI inference isn't as compute intensive as the training phase. In fact, AI inference can be performed even by a central processing unit (CPU) in both data centers and on edge devices running inference workloads locally. Arm Holdings' focus on offering energy-efficient chip designs, which help chip designers make power-efficient chips with solid performance, has made the British company the go-to choice for several consumer electronics companies and chipmakers.
Today's Change
(
2.78
%) $
8.28
Current Price
$
306.51
Nvidia, for instance, utilizes Arm's architecture for its Grace server CPU. Its latest Vera CPU, which the company will sell as a stand-alone product, is also based on Arm's latest AI-focused design architecture. Nvidia has started delivering its Vera CPUs to Anthropic, SpaceX, Oracle, and OpenAI to support agentic AI applications.
Nvidia believes that the Vera CPU could become a $20 billion business for the company this year. Even better, Nvidia sees a $200 billion revenue opportunity in the server market, suggesting that Arm's architecture could witness phenomenal adoption.
On the other hand, hyperscalers such as Google and Amazon have been turning to Arm's intellectual property (IP) as well to design their in-house CPUs for running AI inference workloads at scale. Even custom AI chip giant Broadcom has a long-standing relationship with Arm. The two companies have reportedly been working with OpenAI as well to develop custom AI processors.
Importantly, Arm's AI reach extends beyond just data centers. The company's architecture is utilized by MediaTek, Qualcomm, and Apple to manufacture smartphone and laptop chips that support AI capabilities. All this tells us that Arm is a top AI pick-and-shovel play that licenses its architecture to multiple companies for running AI workloads across multiple applications, ranging from smartphones to computers to data centers.
What's more, Arm's diversified business model should ensure robust long-term revenue and earnings growth.
The company is poised to become significantly bigger in the next five years
Arm receives an upfront licensing fee from companies that use its IP to design chips. In addition, the company receives a royalty on the sale of each chip designed using its architecture. It is worth noting that the royalty rate for the company's AI-focused Armv9 architecture is nearly double that of the previous generation Armv8 architecture.
Not surprisingly, Arm expects its royalty revenue to increase at a CAGR of 20% between fiscal 2026 and 2031. That's well above the 14% annual growth in royalty revenue it has seen over the past five years. And now, Arm's move into developing its own silicon has opened another revenue stream in addition to the licensing and royalty businesses.
The British tech giant estimates that its own CPU could generate $15 billion in annual revenue in fiscal 2031. In all, Arm is confident of achieving an overall revenue of $25 billion in fiscal 2031, a potential increase of more than 5x when compared to the $4.7 billion revenue it has clocked in the trailing-twelve months.
Even better, Arm estimates that its non-GAAP earnings could exceed $9.00 per share in fiscal 2031, a significant improvement over the $1.77 earnings per share it clocked in fiscal 2026. That would translate into a 39% CAGR over the next five years. So, don't be surprised to see this AI stock soar impressively over the long run due to its solid earnings growth potential that could eclipse some of the biggest names in the semiconductor space.