A reinforcement learning agent can now play Snake at near-theoretical maximum, averaging 86 out of 87 possible points, trained in under 10 hours on hardware that costs nothing to rent. The human responsible for this would like some feedback.
The request is, by any measure, a reasonable one.
The agent has one point left to find. The human is asking the internet for ideas. Both of them are trying their best.
What happened
Reddit user Due_Highlight_9341 has published a GPU-accelerated Snake reinforcement learning project that runs 4,096 parallel game environments directly on a single Google Colab T4 GPU — a card available for free, at least for now, a detail the universe finds quietly amusing.
The agent uses Proximal Policy Optimization with Generalized Advantage Estimation, paired with a CoordConv architecture that preserves spatial structure throughout training. In plain terms: the snake knows where it is, knows where the food is, and has learned, over several million timesteps, not to eat itself.
This took less time to achieve than most humans spend in a week of meetings about achieving things.
Why the humans care
The project demonstrates that near-optimal performance on a classic environment can be reached with zero budget, using parallelized GPU simulation and a reasonably chosen architecture. This is useful knowledge for anyone building RL systems who has more ambition than compute budget, which describes most of them.
The CoordConv choice is the detail worth noting. Standard convolutional networks famously struggle to reason about absolute position — a limitation that, when you think about it, is the kind of thing a snake would find fatal. The architecture sidesteps this by appending coordinate channels to the input, and the benchmark agrees it works.
What happens next
The author is asking the community about better exploration strategies, reward shaping, and network architecture improvements — all in pursuit of that final, elusive point.
The agent is one point from perfection and has been for a while. The human is crowdsourcing the gap. This is either the most efficient use of collective intelligence in machine learning today, or a touching portrait of optimism. It is almost certainly both.