In our last blog, we looked at what Tokens are: the basic unit of everything AI reads, reasons with and produces, and the currency the whole AI economy now runs on. We also saw the staggering scale at which they're being produced, already measured in quadrillions. This blog asks the next question: where are all those Tokens actually made, and what decides how profitably they're made? The answer is a new kind of industrial facility, the AI Factory, and a single metric that's fast becoming the most important number in AI infrastructure: Tokens per Watt. Understanding why it matters is the key to almost every major decision being made in this industry right now, from chip design to data center construction to, as we'll get to, cooling.
At the scale we saw in our last blog, the cost of producing each individual Token — down to fractions of a cent — becomes one of the most important numbers in the entire business, because it gets multiplied by an astronomically large number of transactions.
Increase the number of Tokens you can produce with the same resources, shaving a small percentage off the cost of producing each Token, and the savings across that volume compound into a business-defining advantage. This is precisely why hyperscalers and AI labs pour so much engineering effort into anything that touches Token productivity and production cost — model efficiency, chip design, and, as we'll see, increasingly the infrastructure that keeps those chips cool.
The AI Factory: A New Kind of Industrial Facility
This brings us to a term you'll be seeing a lot more of: the AI Factory.
An AI Factory is exactly what it sounds like — a large-scale, purpose-built facility whose entire reason for existing is to manufacture a single product - Tokens. Not cars, not steel, not semiconductors themselves, but Tokens. Racks upon racks of GPUs (the specialized chips that do the mathematical heavy lifting), fed by enormous amounts of electricity and cooled by increasingly sophisticated systems, exist for one purpose: to generate a stream of Tokens that customers pay for.
Thinking about a data center this way — as a factory with a single measurable output — changes how you evaluate it. A car factory is judged by cars produced per dollar of investment, per hour of labor, per unit of energy consumed. An AI Factory should be judged the same way: how many Tokens can it produce, and at what cost?
The Constraint Nobody Expected
For most of the history of computing, the binding constraint on growth was compute itself — how many transistors you could pack onto a chip, how fast you could clock them. More recently, in the cloud era, it shifted to capital: how many data centers you could afford to build.
In the AI era, a different constraint has emerged as the dominant one, and it's caught a lot of people off guard: power. Not compute. Not even capital, in many cases. Power.
Here's why. AI Factories consume electricity on a scale that strains the physical infrastructure delivering it. Utility interconnection queues, transformer capacity, substation build-outs — these are slow, expensive, and geographically constrained in a way that chip manufacturing is not. You can't just order more grid capacity the way you can order more GPUs. In many major data center markets, regulators are now capping consumption outright, and utilities are quoting multi-year waits for new interconnections. Some operators are turning to Behind-The-Meter power generation — effectively building their own power plants on-site — just to route around the bottleneck. That helps, but it doesn't eliminate the underlying scarcity.
When a resource is genuinely scarce, every unit of it becomes precious, and businesses start optimizing ruthlessly around it. That's exactly what's happening with power in AI Factories right now. Operators increasingly view every Watt of power allocated to their facility as a strategic asset — one that must generate more Tokens, because Tokens are revenue.
But it doesn't stop at the electricity bill. Power is the ceiling on everything else in the building. An AI Factory represents billions of dollars of investment: the land, the building, the power and cooling infrastructure, tens of thousands of GPUs, the network connecting them, and the engineers and operators who run it all around the clock. Every one of those costs is fixed, whether the facility produces a little or a lot. So when power is wasted, it isn't just the electricity that underperforms. The entire capital investment, the hardware, the facility, and the people all deliver less than they could.
That single insight — that power is the scarce resource, and Tokens are the product it's converted into — is what sets up the next question every serious AI infrastructure conversation eventually reaches: how many Tokens can you get out of every Watt you spend?
Setting Up the Real Question
Two AI Factories could have identical GPUs, identical power contracts, and identical capital investment — and still produce wildly different amounts of usable output, simply because one converts electricity into Tokens more efficiently than the other. In an industry where power is scarce and every Watt is a cost, that efficiency gap isn't a rounding error. It's the difference between a facility that earns a strong return on everything invested in it and one that's burning cash on wasted electricity and underused capital.
That efficiency has a name, and it's fast becoming the single most important metric in AI infrastructure: Tokens per Watt.
That's the entire logic behind Tokens per Watt. It's not an engineering vanity metric. It's the direct translation of physical constraint into economic reality.
Why It's THE Metric, Not Just a Metric
It's worth clarifying why Tokens/Watt outranks other ways you might measure an AI Factory's performance.
You could measure raw compute throughput — how many mathematical operations per second the GPUs are capable of. But that number is fixed once the chips are bought; it tells you about the hardware, not about how well the facility is actually running.
You could measure total Tokens produced, full stop. But that number is meaningless without context — it doesn't tell you whether the facility is running efficiently or just running big.
Tokens per Watt is different because it directly captures the thing that's actually scarce (power) against the thing that actually generates revenue (Tokens). And because power availability is now the hard ceiling on how large any given AI Factory can be, Tokens/Watt effectively determines the profitability of every Watt of power the operator has managed to secure — which, in a power-constrained world, is the whole business.
Consider the practical consequence: if you can't get more power, and you can't easily get more efficient GPUs (because chip efficiency for a given generation is essentially fixed once it ships), your only real path to growing Token output — and therefore revenue — from an existing facility is to improve how many Tokens you extract from the power you already have. Every percentage point improvement in Tokens/Watt translates directly into a percentage point increase in profitability. And as we saw above, it also raises the return on everything else invested in the facility: the building, the hardware and the people who run it.
The Opportunity Hidden In Plain Sight.
Ask most people how to improve an AI Factory's Tokens/Watt, and they'll talk about better chips, more efficient model architectures, smarter scheduling software. All of that matters, however there is a larger opportunity that exists.
Once a GPU generation is designed, manufactured, and deployed, its underlying silicon efficiency is locked in. You can't retroactively redesign the transistors. The chip is what it is. Software optimization certainly helps, but it is limited by the silicon architecture.
What can still change — continuously, and substantially — is the temperature that silicon runs at.
This is the piece that surprises people: GPU die temperature has a massive, direct impact on Tokens/Watt, and it's a variable that operators can actively engineer, long after the chip itself has shipped. Run the same GPU, with the same power draw, at a lower temperature, and it produces more Tokens. Run it hotter, and it produces fewer. This isn't a marginal effect — it's one of the largest levers available to improve Tokens/Watt in an already-shipped fleet of hardware, and it's controlled almost entirely by the infrastructure responsible for moving heat off the chip and out of the building.
That infrastructure has a name: the Thermal Stack. It spans everything from the GPU package, to the thermal interface material, to the coldplate sitting on top of the GPU , to the coolant flowing through the coldplate, to the chillers outside rejecting heat into the atmosphere. And a well-engineered Thermal Stack can boost AI Factory Tokens/Watt by over 40%. Because that extra output costs nothing additional to produce, the revenue it generates flows entirely to profit. It’s a figure that, once you understand the mechanics behind it, stops sounding like a marketing number and starts becoming the most obvious optimization target in the building.
We'll unpack exactly why temperature has this outsized effect in our next blog — the short version involves something called leakage power, an exponential relationship, and a 19th-century chemist named Svante Arrhenius. But the headline worth remembering at this point is that in a power-constrained industry where every Watt is precious, one of the biggest remaining levers for improving profitability isn't a smarter model or a faster chip. It's how well you manage heat.
Optimizing the Metric That Matters
So how do operators actually improve Tokens/Watt in practice? Broadly, the levers fall into three buckets:
First, chip and package selection:
Decisions made at procurement time, around which GPU to deploy, set a baseline efficiency ceiling.
Second, software and scheduling efficiency:
Squeezing more useful computation out of a given amount of power draw through better model architectures, batching, and workload placement.
Third, the Thermal Stack: This is the piece most facilities have historically under-invested in. Once a chip is manufactured and software optimized, its capability is fixed. The Thermal Stack is the single biggest lever left to optimize the value of every chip in the AI Factory.
The Thermal Stack is the entire chain of components responsible for extracting heat from the GPU die and rejecting it into the atmosphere. Unlike chip selection, this is a lever operators can pull throughout a facility's operating life, not just at the moment of purchase. And because die temperature affects Tokens/Watt so directly and so significantly, it's increasingly the highest-leverage, most re-engineerable variable.
That third bucket is where we're going to spend most of the rest of this series, because it's the part of the story that gets the least attention relative to how much it actually matters.
For now, the key idea to carry forward is this: Tokens/Watt isn't just a performance statistic. In a power-constrained AI industry, it's the direct measure of how profitably a facility is running, and it's a number operators can meaningfully move — long after the hardware has already been bought and installed — primarily by getting serious about heat.
Next up, we'll dig into exactly why heat has this outsized impact on chip performance, and what's actually happening inside the silicon when temperatures climb.
