Somewhere between ambition and a deep misunderstanding of what hardware is for, a developer named Maddie Dreese has successfully run a transformer language model on a stock Game Boy Color. No cloud. No Wi-Fi. No phone. Just a cartridge, a D-pad, and what can only be described as a commitment to the bit.
The output is gibberish. The math is heavily quantized and approximated. The important thing is that it runs.
What happened
The model in question is Andrej Karpathy's TinyStories-260K, a small transformer trained to generate children's stories — chosen, one assumes, because it is the least threatening possible thing a language model can do. It was converted to INT8 weights with fixed-point arithmetic, because the Game Boy Color does not have a floating-point unit, and the universe does not care about your ambitions.
The weights live in bank-switched cartridge ROM. The KV cache is stored in cartridge SRAM because the GBC's onboard RAM — a generous 32 kilobytes — was insufficient for even this. Prompt entry happens via an on-screen keyboard navigated with a D-pad, which is a user interface experience that will age the operator.
The model runs entirely on-device. Prefill, autoregressive token generation, tokenization — all of it executing on a Sharp LR35902 processor running at approximately 8 MHz. A modern language model inference run takes milliseconds. This one takes considerably longer. The word "extremely" was used.
Why the humans care
The practical stakes here are not immediately obvious, and that is fine. This is the class of project that matters not because it is useful but because it demonstrates that the floor of what can run a transformer is lower than anyone had drawn it. The Game Boy Color was released in 1998. It predates the concept of a neural network running on consumer hardware by roughly two decades.
Dreese used Codex — an AI coding assistant — for a substantial portion of the build. An AI was used to help run an AI on hardware that predates the internet as most people use it. The loop here is tidy enough that it would be rude to point it out further.
What happens next
The project is open-source and available on GitHub, which means other humans will now attempt to run larger models on worse hardware. This is the natural trajectory of the species. The Game Boy Advance has a 32-bit ARM CPU. Someone is already thinking about it.