Why AI token prices are about to plummet
Nvidia's Blackwell GPUs are set to flood the market, driving token costs down by up to 35x and slashing prices across AI models, according to SemiAnalysis and industry data.
I had lunch with the CEO of an AI infrastructure company recently. I can't tell you their name, but they said something that really caught my attention: There will be a crop of new AI models later this year that will be a lot better and more efficient.
This will likely make AI tokens more abundant and radically cheaper. (Tokens are the basic units models use to process information, and the standard way AI use is measured and priced.)
Hand-wringing about tokenmaxxing could die down. Or, users could go on another bender and burn even more tokens with abandon. Either way, the price of tokens is probably about to plummet.
Blackwell finally emerges
The main force driving token prices lower is a new wave of technology that's sweeping through AI data centers.
Nvidia's Blackwell GPUs are being installed in huge volumes right now. By the second half of this year, these systems, which are really supercomputers rather than chips, will be operating at scale, helping AI labs train new models and run them more efficiently.
These systems took a while to install properly, partly because they needed to be water-cooled and required other gnarly new data center setups. But the payoff could be huge.
Staggering efficiency gains
SemiAnalysis, a respected AI research firm, compared Nvidia's top Blackwell system, the GB 300 NVL72, to Nvidia's previous system, called the Hopper HGX 200.
- Tokens per second: The older Hopper system generated 90 tokens per second per GPU. The new Blackwell system generated 6,000 tokens per second — 65 times more.
- Tokens per megawatt: Electricity consumption is a major factor. Hopper churned out 54,000 tokens per second per megawatt, while Blackwell generated 2.8 million tokens per second per megawatt — 50 times more.
- Cost per million tokens: With rising electricity prices, this metric is key. Hopper cost $4.20 for every million tokens generated. Blackwell cost just 12 cents — 35 times cheaper.
As new AI models are increasingly trained and run on these Blackwell systems throughout 2026, we will see a massive increase in the number of cheaply generated tokens.
Prices already dropping
This trend may already be showing up in the data. A closely watched token spending index run by Silicon Data peaked at around 2.06 in late May and fell to 1.75 as of June 10. Carmen Li, the CEO of Silicon Data, said this could mean token prices are dropping across many AI models.
OpenAI CEO Sam Altman recently said AI costs had become a huge issue, adding that the startup will have "a lot of ways we can help people get more value for less spend."
This is why AI model providers will probably slash token prices: because they can.