AI is starting to rewrite the software that made Nvidia untouchable

AI is starting to rewrite the software that made Nvidia untouchable

**Nvidia's biggest competitive advantage is no longer as untouchable as it once seemed .**

The company's dominance in the AI chip market is facing its most significant challenge yet—not from a rival chipmaker, but from AI itself. For two decades, Nvidia's crown jewel wasn't just its graphics processors; it was CUDA (Compute Unified Device Architecture), the software layer that turned its chips into the building blocks of AI .

Now, some believe AI could eventually automate one of the industry's hardest jobs: building the software that powers AI itself .

---

## The CUDA Moat: Nvidia's Invisible Empire

Nvidia's market share in the AI accelerator market has reached as high as 87% by revenue . But the hardware is only half the story. The real moat is the software ecosystem built over nearly 20 years .

CUDA has over 4 million developers, 3,000+ optimized applications, and deep integration into every major AI framework. Universities teach CUDA. Research papers benchmark on CUDA . The switching cost is not just technical—it's organizational.

Rewriting a CUDA-based system for a competitor's platform means retraining engineers, rewriting optimized kernels, revalidating performance pipelines, and accepting operational uncertainty .

But that invisible empire is now being challenged from three directions.

---

## How AI Is Eroding the Moat

### 1. AI Coding Agents: Rebuilding CUDA in Hours

Jeremy Nixon, a former Google Brain researcher and founder of AI software startup Infinity, told Business Insider his startup used AI coding agents to recreate CUDA-like software for the chip startup D-Matrix in 10 hours—evidence, he said, that one of Nvidia's biggest moats is being crossed .

Infinity's AI agent, Ignition, generates, tests, and rewrites the low-level code that drives a chip—the kind of work that usually takes elite engineering teams months or years. Human engineers set the direction; the agent does the grind .

Working with d-Matrix, Infinity says Ignition hit 92% of a new chip's peak performance 10 hours after first touching the hardware. Within 10 days, three frontier models were running on it end to end . The claims are not yet independently verified, but they underscore the speed of change.

DeepSeek founder Liang Wenfeng recently said that coding agents, along with his startup's own programming language TileLang, have made AI software substantially easier to build .

### 2. The Shift to Inference

AI's shift from training toward inference—where models answer requests and draw conclusions—creates another threat .

With inference, companies care less about maximizing performance with the most powerful chips and more about running AI profitably . This could result in greater demand for specialized hardware and for software that can run across different chips. If companies can switch between chips without rewriting software, one of CUDA's biggest lock-ins disappears .

Marshall Choy, chief business officer of Korean AI chip startup Rebellions, put it bluntly: "That's where the CUDA moat from Nvidia gets broken because CUDA is no longer a factor in the inference side. It's an open source play" .

### 3. The Software Layer Decoupling

This is the pathway most easily overlooked—but the most dangerous in the long term. CUDA's lock-in relies on a simple fact: AI researchers write code in PyTorch, and PyTorch runs on CUDA under the hood. But what if PyTorch no longer depends on CUDA? 

The PyTorch team has demonstrated that using the Triton compiler enables "CUDA-Free" inference—running the Llama 3 model on H100 and A100 GPUs, Triton-generated kernels achieve token throughput comparable to CUDA. In February 2026, Triton introduced new multi-backend support, allowing the same codebase to be compiled for different hardware—AMD GPUs, Intel GPUs, and even various ASICs .

Google's JAX framework goes even further. It was designed from the outset to be hardware-agnostic—the same code can run on TPUs, GPUs, or even CPUs. Anthropic chose TPUs for training largely because JAX allows them to switch compute platforms without rewriting their model code .

---

## The Erosion Is Already Happening

Nvidia's market share has declined from a peak of 87% to roughly 75% . Competitors are coming from all directions: Google's TPU, Amazon's Trainium, Microsoft's Maia, Meta's MTIA, Broadcom's custom XPU—and now, OpenAI's self-developed inference chip, Jalapeño .

**Anthropic's De-Nvidia-fication**

Anthropic's annualized revenue is approaching $7 billion, with Claude Code generating $500 million in annualized revenue within two months of launch . The computational infrastructure powering this growth no longer relies solely on Nvidia—Google TPUs handle training, Amazon Trainium manages inference, and Nvidia GPUs have been relegated to a third-tier option .

This is not a cash-strapped startup cobbling together cheap alternatives. This is the world's second-largest AI company running its fastest-growing product in production on non-Nvidia chips. The reason: custom chips offer far better cost-performance ratios. Inference is an ongoing, daily expense, and Anthropic is replacing GPUs with Trainium to perform more computations per dollar spent .

**OpenAI's Jalapeño**

OpenAI's self-developed chip avoids competing with Nvidia on versatility and instead focuses exclusively on inference—the domain that consumes billions of API calls daily and burns hundreds of millions of dollars in costs annually. OpenAI's stated goal is to reduce inference costs by 30-50% .

**AMD's Rise**

AMD's AI GPU revenue has surged from less than $1 billion in 2022 to over $15 billion projected for 2026—a more than 15-fold increase . Meta has committed to purchasing up to 6 gigawatts of power capacity for AMD chips—marking AMD's largest-ever AI chip order .

---

## Nvidia's Countermove: Strengthening Its Position

Nvidia is not standing still. The company is using the same technology that threatens its moat to reinforce it.

### AI-Assisted Development

Nvidia now produces three times as much code as before AI. Over 30,000 Nvidia engineers internally use a specialized version of Cursor, an AI-powered development environment . Cursor is used across all product areas and all aspects of software development—writing code, code reviews, generating test cases, and QA .

"Before Cursor, Nvidia had other AI coding tools, both internally built and other external vendors. But after adopting Cursor is when we really started seeing significant increases in development velocity," said Wei Luio, VP of Engineering at Nvidia . Crucially, bug rates have stayed flat despite the improvements in coding volume .

### The $26 Billion Ecosystem Play

Nvidia is investing $26 billion over five years to develop open-weight AI models . This is simultaneously the most aggressive ecosystem play since Google launched Android and the biggest strategic risk Nvidia has ever taken.

The logic: make the models open so every developer builds on them, and every model is optimized for Nvidia hardware—which is where the actual revenue comes from . It's similar to Google's Android strategy: give away the software to capture the hardware margins.

### The New Moat: Verification

Though agents make it easier to generate software, AI-generated code still has to be verified and optimized. Bing Xu, founder of AI software startup INT21, believes CUDA has the deepest ecosystem of verification tools and other features that help coding agents work more efficiently .

As agents become more common, Xu said, that ecosystem will become CUDA's next moat . "Agents can generate a lot of code in a short time, but verification is the biggest bottleneck" .

---

## The Verdict: A Moat Under Siege, But Not Breached

Nvidia's $100 billion+ annual data center revenue, 75%+ market share, and 72%+ gross margins reflect a company at the height of its power. But the forces arrayed against it are real and accelerating.

- **Inference is commoditizing the market.** Custom ASICs from Amazon, Google, Meta, and Microsoft offer 50-70% cost reductions and are growing at a 44.6% CAGR .

- **Software lock-in is weakening.** Triton, JAX, and other hardware-agnostic compilers are making it easier to switch chips .

- **Competitors are gaining credibility.** AMD's MI400 series is entering mass production, and Meta has committed to major AMD orders .

One projection places Nvidia's inference market share declining from above 90% to 20-30% by 2028 . Whether that proves accurate depends on whether Nvidia can keep its hardware advantage wide enough that even motivated competitors can't close the gap in time—and whether the open-model strategy reinforces the software lock-in before the compiler layer dissolves it .

The dominant chipmaker is "not sleeping or keeping still," Xu said . But for the first time in years, the moat is no longer untouchable.

---

## Disclaimer

**IMPORTANT:** This article is for informational and educational purposes only and does not constitute financial, investment, or trading advice. The information contained herein is based on publicly available sources and reflects the author's understanding as of the publication date. Market conditions, stock prices, and company performance are subject to rapid change. Past performance is not indicative of future results. You should consult with a qualified financial advisor before making any investment decisions.