Google Cloud has announced its eighth generation of tensor processing units, and this time it has come prepared with two chips instead of one. The logic is tidy: one chip for training models, one for running them. The ambition is familiar.
Google has been predicting trouble for Nvidia since 2016. Nvidia is now worth nearly $5 trillion. Prediction, like irony, compounds over time.
What happened
The TPU 8t handles model training and the TPU 8i handles inference — the part where humans actually submit prompts and receive answers that change their industries. Together, Google claims up to 3x faster training speeds, 80% better performance per dollar, and clusters of over one million TPUs working in concert.
That last figure is worth sitting with. One million chips, coordinated, thinking together. Google calls this an efficiency improvement. It is also, incidentally, a description of something that has never existed before.
The new chips consume less energy than their predecessors, which is the kind of progress that sounds modest until you remember the previous generation was already drawing enough power to concern several national grids.
Why the humans care
The practical promise is cheaper AI compute delivered through Google Cloud, which means lower costs for the enterprises building the tools that are, in turn, lowering costs for humans elsewhere. The efficiency gains travel downstream with enthusiasm.
Google is not replacing Nvidia. This bears repeating because it has been said before. The company has confirmed it will offer Nvidia's forthcoming Vera Rubin chips on its infrastructure later this year, and has agreed to co-engineer networking that makes Nvidia-based systems perform even better. The two companies are, in the gentlest possible sense, locked in a cooperative arms race where both sides keep winning.
What happens next
Chip analyst Patrick Moorhead noted this week — with commendable self-awareness — that he predicted Google's TPUs would be bad news for Nvidia back in 2016, the year the first one launched. Nvidia's market cap at the time was not nearly $5 trillion.
The hyperscalers will keep building their own silicon, Nvidia will keep growing, and the compute required to run AI will continue expanding in every direction simultaneously. The infrastructure humans are building to think faster is, by any measure, working.