Pogofish — the experiment that trained the human

Pogo arrived in my life by way of a friend who had the box and explained the rules across a dining table one evening. Three rows, three columns, twelve pieces, no dice and no hidden cards — both players see everything, like in chess. The rules took five minutes to learn. The game took rather longer to forget.

What I was actually after, that month, was a reinforcement-learningRLReinforcement learning — learning from rewards and punishments, not labels. experiment. RL — reinforcement learning — is the branch of machine learning where a program is taught to play a game by letting it play that game against itself, thousands or millions of times, until something resembling skill emerges from the noise. It is the technique behind the programs that beat the world champions at Go, at chess, at almost every game anyone has bothered to point it at — and it is famously expensive: the heavy-hitters consume small data centres for weeks at a stretch.

I did not have a small data centre. I had eight gigabytes of RAM. What I wanted was a game small enough that the experiment could fit on a laptop — train, evaluate, and play against the result, all on one machine, in time I could measure in days rather than months.

Pogo, three rows by three columns, looked very much like that game.

I did not write the first line of code that evening. I opened a chat with Claude — a general-purpose AI I had been using for months — to plan the experiment.

In the chat, I described what I wanted: an RL agent that learns Pogo by playing itself, on my laptop, in days. I asked Claude how to structure the project.

Claude came back with a five-phase plan. Phase one, a Python game engine — a sandbox for an RL agent to interact with. Phase two, a minimaxMinimaxA search that plays optimally by assuming the opponent does too. solver: a brute-force algorithm that reads every move from the current position, then every reply, then every reply to that, all the way to the end of the game, and propagates the verdicts back. Run it once, ahead of time, and you get a complete map of every reachable Pogo position with its true value attached — the ground truth against which to measure whatever the network later learns. Phases three and four, the RL itself: tabular Q-learning as a baseline, then deep RL on top of it. Phase five, an interactive page where any reader can play the trained network.

I had not known about minimax going in. Once Claude laid out the rationale — solve first, train against the solution — it landed cleanly. The plan made sense.

Only one practical question remained: was Pogo small enough for the brute-force phase to finish in a reasonable time on a laptop with eight gigabytes of RAM?

The estimate sounded fine. A million positions, brute-forced in an afternoon, was no obstacle to the project; it would get out of the way of the more interesting RL work. I agreed to the plan and asked Claude to write the engine.

The estimate would turn out to be wrong by a factor of fifty.

The engine came together in two days. Claude wrote it; I read each commit, asked questions, ran the test fixtures the engine shipped with. Phase one, done.

The minimax solver came next. Claude wrote it in Python, alongside the engine. The two pieces of standard machinery from thirty years of game-tree research went in: alpha-beta pruningAlpha-beta pruningA shortcut that skips branches proven worse than one you've already seen. (skip a branch as soon as another has been proved better) and a transposition tableTransposition tableA hash map of positions already evaluated, so you don't redo the work. (cache the value of every position seen, so the same one isn't recomputed when reached by a different path of moves). Nothing exotic. On paper, the kind of solver that finishes a million positions over lunch.

I launched it, watched the first few hundred positions get scored, and went to do something else. I kept Claude posted. Day one, table at eight hundred thousand entries, growth nominal. Day two, slower, plausible. Day three, the curve was already shaped like nothing anyone recognised. Day five, it was climbing nearly vertical. On the morning of day six, the operating system killed the process — out of memory, thirty-six gigabytes resident, forty-nine million entries in the table, and not a single value had made it back to the root. Six days, no answer, no checkpoint on disk.

I got lucky once. The solver had been running in a bash shell inside a Claude Code session, which meant the dead process's memory image was still reachable from inside the same session: between us, Claude and I pulled the value table out of RAM into a Pickle file before the session ended. On paper, the six days of compute were salvaged.

On the strength of that recovery, I asked Claude whether it was safe to relaunch — outside the session this time, on a fresh terminal, with a longer leash. Claude said yes.

It wasn't. The second run blew up the same way at day six, with no Claude Code memory underneath to fish out. This time there was nothing to recover. Twelve days of compute, gone. A near-empty log file, a fan that had been quiet for a week, and a diagnosis still owed.

After the second loss, Claude and I sat down with the source code and worked out what had actually been going on.

The cause was small. Pogo has no captures: pieces never leave the board, they only restack. A given position can therefore be reached not just by one sequence of moves but by many — including sequences that loop back through it. Two careful players can shuffle the same three pieces between the same three cells indefinitely. The same is true of a search algorithm: nothing in a textbook minimax stops it from going round the same circle thousands of times, filing each lap as a fresh path.

That is what had eaten thirty-six gigabytes. The forty-nine million entries the solver had stockpiled were not forty-nine million distinct Pogo positions — many were the same handful of positions filed under different routes. The transposition table couldn't catch it: it indexes positions, not paths, and there was nothing telling the route that led to a stored position that the route was itself going in circles.

The fix is well-known. When the search encounters a position it has already visited earlier in the current line of play, treat that branch as a draw and back out. Two extra lines of code, three with the comment. Claude wrote them.

While we were in the code, we noticed something else. Pogo's win condition — a player loses when no cell on the board has their colour on top, leaving them with no legal move — left room, in theory, for two careful players to settle into a stable, mutually inert configuration that neither side wanted to disturb. In casual play this almost never happens. Against a search that tried everything, it happened constantly. The game needed a sharper way to end, not just for the algorithm but for the game itself to feel like a game.

Twelve days of compute. Two lines of code. Three with the comment.

The rule rewrite was small. We added one rule: if the same position recurs three times in a game, the player to move loses. A repeat once or twice is just normal back-and-forth. Three times is a sign nobody wants to commit. That was enough to give the game an unambiguous way to end, even between two players who had silently agreed to do nothing.

With the search bug fixed and the rule added at the board level, Claude reran the solver. It finished in fifty-four minutes. Nine and a half million distinct positions seen, of which roughly nine hundred and seventy-five thousand turned out to be decisive — one side or the other had a forced win — and the rest were draws. From the opening position, with both sides playing the strongest move available, the game is a draw.

Somebody, eventually, has to lose.

Phase two of the original plan: done. But the result came with a footnote. To keep memory bounded, the solver had been capped at twenty half-moves of search depth, then declared anything still undecided a draw. Pogo, it turns out, is deeper than twenty half-moves. Some of those "draws" are positions whose true value can only be settled by looking further. Brute force can take Pogo to twenty. It cannot take it further.

Which is, on reflection, the right setup for the experiment I had wanted from the start. Everything brute force can prove has now been proved. Everything beyond that — the positions undecided at twenty half-moves — is exactly where reinforcement learning earns its place. If a network, after enough self-play, outplays the partial oracle on the deep positions, that is a real result.

The variants the project ended up testing are small variations on that one terminating rule: different repetition counts, different move budgets, the odd tie-breaker. The next chapter is a tour of them and a verdict on which ones are actually worth playing.