ARM, RISC-V, and AI

Perhaps it was written in the stars: on the same day Alibaba’s T-Head XuanTie C950 was announced — benchmarking at 22 points per GHz, running at 3.2 GHz — ARM unveiled its very first in-house SoC. Two trajectories began to intersect, and the war between RISC-V and ARM finally started to show real smoke.

Anyone in the semiconductor industry knows that launch events are full of misdirection, and every decision involves interests and calculations that outsiders cannot fully see. What follows is my personal read.

1. What does RISC-V actually have going for it in the AI era?

A common pushback goes like this: AI is just matrix multiply. It runs on custom instructions and hardware accelerators, or on vector units — all of which ARM and x86 already have. So where is RISC-V’s edge?

That is exactly the right question. Purely on compute, RISC-V has no advantage today. Software and hardware optimization maturity are also well behind ARM and x86. But here is the thing: at least on the inference side, AI workloads are almost never compute-bound anymore. The real bottleneck is memory bandwidth and communication latency. (Training is still heavily compute-bound, but inference is where most systems need continuous optimization in practice.)

You either go the Cerebras route (wafer-scale SRAM) or the Groq route (deterministic streaming execution, no cache hierarchy, static scheduling — their TSP architecture), both aggressive but effective. Or you accept that feeding your compute units and reducing latency requires accelerating memory movement and communication first.

The main strategies for this: prefetch, compression, DMA offload. Either you hardwire the logic, or you use a cluster of small CPUs to handle data movement. Given how quickly AI models and data formats evolve, keeping things flexible is critical — and a mesh of small, programmable cores is the smartest way to stay flexible. This is not just theory: Cambricon, Biren, and Tenstorrent are all shipping AI chips that use RISC-V cores for control and data movement today.

Now you also want to extend those cores with custom instructions for collective communication ops. At this point you face a choice: ARM or RISC-V? ARM has a mature software ecosystem, but you pay for a license and you cannot freely customize the ISA. RISC-V is open source, free to download and modify, no royalties. The software ecosystem is thinner, but it is more than sufficient to run data movement kernels. When you lay it out this way, the answer becomes obvious.

2. Why is ARM competing with its own customers?

ARM’s first in-house SoC prompted a lot of people to ask: why would ARM start competing with the very companies that license its IP?

My read: this is not really about grabbing market share. It is about survival.

RISC-V is riding the open-source ecosystem wave. Software and tooling are maturing steadily. It has already started to eat into ARM’s mid-range market, and challenging the high end is a matter of when, not if. ARM’s core business model is under pressure, and in the high-end AI hardware cycle, it is hard to see how IP licensing alone captures a large slice of the value. Customers who can afford to build their own silicon can always pivot to RISC-V.

Not this year. Maybe not next year. But at some point, ARM’s largest customers will have built up enough internal capability and confidence to stop paying the ARM tax and switch. If ARM’s leadership did not see this coming, they would be failing in their jobs. The fact that they clearly do see it means the only rational move is to take a seat at the table themselves — cut into the value chain directly, rather than watch it erode from the outside.

There is also a deeper structural contradiction here. ARM’s entire valuation logic is built on being the pickaxe seller — collecting royalties across the whole semiconductor food chain while Qualcomm, MediaTek, and Apple compete with each other. The moment ARM makes its own SoC, those same companies become both customers and competitors. That relationship is very hard to manage. And the further ARM goes down the SoC path, the stronger the incentive for its customers to accelerate their RISC-V migration. It is almost a self-reinforcing trap.

3. Does RISC-V have a future? What are the real challenges?

Two years ago I genuinely struggled with this question. Today I am unexpectedly optimistic, for two concrete reasons.

First, the number of companies building on RISC-V is growing fast. A few years ago, most startups still defaulted to the playbook of licensing an ARM core and adding a little proprietary logic on top. Now, whether it is a startup or a large company’s internal chip team, RISC-V comes up constantly. When an ecosystem adds more players, and younger ones, the long-run trajectory bends toward it winning.

Second, the infrastructure has matured substantially. I remember picking up RISC-V in 2019 when almost nothing existed — just a Rocket core written in Chisel, an LLVM backend that could not compile Linux, and binaries that only ran on QEMU. Seven years later, development boards are shipping, RVA23 has given the ecosystem a shared design target, IOMMU and PLIC support have matured. RISC-V finally looks like a grown-up platform.

That said, the challenges are real. Three stand out.

Performance and software optimization. Getting a seat at the table is not the same as having influence. Hardware design maturity takes time, and ARM and x86 have decades of co-evolved software and hardware optimization that RISC-V will need to catch up to.

Standardization moves too slowly. Open governance is the foundation of RISC-V’s openness, but the pace of ratification is a serious liability. The Integrated/Attached Matrix Extension Working Group was formed in 2023 — and as of 2026, the spec still is not ratified. That is why T-Head built their own Matrix Multiply Extension (MME) rather than wait. The lesson from RVA23 cannot be ignored here: slow standards processes push vendors toward fragmentation.

Fragmentation and commoditization risk. When RISC-V had little market share, the ecosystem cooperated well. Now that there is real money on the table, the question is whether players will start building their own isolated sub-ecosystems rather than contributing to the shared base. To be fair, RISC-V International is aware of this risk — the RVA Profiles (RVA20, RVA22, RVA23) exist specifically to define a common baseline that keeps different vendors’ implementations compatible. The direction is right, but how well it holds as commercial stakes rise remains to be seen.

This is my first post that leans more explicitly into hardware opinions. All of this is personal view. If you agree, disagree, or just want to trade notes, I am always happy to connect.