Nvidia's Real Moat Isn't a Chip — It's the Software Nobody Talks About

Over the past decade, the entire AI research community has built its tools, libraries, and workflows on CUDA. When a company trains a large language model, the code that orchestrates thousands of GPUs working in parallel is written for CUDA.

Rewriting that code for a rival platform — like AMD's ROCm or Intel's oneAPI — means re-testing every step. It means fixing new bugs and risking delays on projects worth hundreds of millions of dollars.

This lock-in is why Nvidia can charge premium prices and still see customers line up. The company spent $18.5 billion on R&D in fiscal 2026, up 43% from the prior year. A big chunk of that goes into making CUDA better — adding new features, supporting new model architectures, optimizing performance.

More profit funds more R&D, which improves the platform, which attracts more developers, which sells more chips. This cycle is accelerating every year.

The result shows up in the financials. Even with gross margins dipping from 75% to 71.1% — partly because of that $4.5 billion H20 inventory write-down — Nvidia is still printing money at a rate that would make most software companies jealous, let alone a hardware maker.

The non-obvious insight is that AMD's best shot isn't replacing CUDA head-on. It's winning inference (using a trained model to make predictions, not training it) workloads at the edge — the part of AI where a trained model answers questions, generates images, or makes predictions in real time. Inference doesn't require the same deep CUDA integration that training does. It's more standardized and more cost-sensitive. It increasingly runs on lean software where CUDA's extra baggage is a liability, not an advantage.

For a company spending $100 million to train an AI model, the risk calculus is brutal. Even a 5% chance of delay or performance regression isn't worth the savings on cheaper chips. But for deploying that model to millions of users? Cost per query matters enormously, and that's where AMD could carve out real revenue.

The investment implication: Wall Street's AI chip revenue forecasts for AMD are heavily weighted toward training workloads, where CUDA lock-in caps the addressable market. If AMD's growth instead comes from inference and edge deployment (running AI on devices near the user, not in big data centers) — lower-margin, higher-volume work — the revenue mix and profit margins will look very different from what the stock price assumes today.

What would change the picture entirely? A top-10 AI lab publicly running a flagship training job on ROCm without performance penalties. Until that happens, AMD's AI story is an inference story — and investors should price it accordingly.

In fiscal 2026, Nvidia's largest direct customer accounted for 22% of total revenue. The second-largest accounted for 14%. These are almost certainly hyperscale cloud providers — the exact companies building their own chips.

So why do they keep buying? Why? Because their business customers demand cloud servers that run CUDA. When a bank or a pharma company rents cloud GPUs to train an AI model, their engineers write CUDA code. If a cloud provider only offered custom chips with a different software stack, those enterprise customers would go to a competitor that offers Nvidia.

Custom chips (called ASICs) trade flexibility for efficiency. They're great at narrow tasks — like running a specific model in production. But they can't replace general-purpose GPUs for the messy, experimental work of training new models. Cloud providers will use their custom chips for internal workloads, but they'll keep buying Nvidia for the instances they sell to everyone else.

This setup means Nvidia profits from the big cloud companies' reach — even as those same companies build their own rival chips. Big cloud companies keep buying Nvidia chips because their business customers demand CUDA support. That happens even as those same cloud giants build their own custom chips.

Computer science programs around the world teach GPU programming using CUDA. When a PhD student spends four years writing thesis code in CUDA, she carries that expertise into her new employer — which then builds on CUDA because it's what the team already knows.

This creates a self-reinforcing cycle through the labor market. Companies hire CUDA-trained engineers, which means their projects run on Nvidia hardware, which means the next generation of students learns CUDA to be employable.

This human-capital lock-in may be even harder to break than the technical lock-in described in domino one. You can rewrite code. You can't easily retrain an entire generation of AI researchers.

Nvidia sits on $62.6 billion in cash and investments (as of fiscal year end 2026). Last fiscal year alone, the company poured $17.5 billion into private companies and infrastructure investments. Some of that money flows into university partnerships, research grants, and developer programs. That deepens the talent pipeline, which feeds right back into more CUDA adoption.

When the U.S. Restricts chip exports to China, the conversation focuses on hardware. But CUDA is the software those chips need to be useful. Controlling it means controlling the programming tools, the performance tuning, and the entire developer ecosystem.

Nvidia's H20 chip — designed as a lower-performance product for the Chinese market — generated only about $60 million in revenue in fiscal 2026, after the company took a $4.5 billion inventory charge on unsold units and purchase commitments. International revenue overall was 31% of Nvidia's total.

Tightening export controls or foreign governments pushing for AI sovereignty could create headwinds for Nvidia's international business. China and the EU both have incentives to reduce dependency on a single U.S. Company's proprietary software stack.

But building a CUDA alternative isn't just a technical challenge — it's an ecosystem challenge. And ecosystems take decades to build. This is a real long-term risk, but not one that changes the story in the next two to three years.