I replicated the vision-only Pokémon run Anthropic showcased on the Fable 5 launch page — then I read its thinking chains

20 Jul, 2026

Anthropic says Fable 5 can play Pokémon from vision alone. I verified that claim independently: 1,785 decisions and $65.40 to the first gym badge — then I read its thinking chains in the logs, and the result stopped looking so simple.

In short — Anthropic's Fable 5 launch page claims the model beat Pokémon FireRed "with a minimal, vision-only harness." No protocol, no step count, no cost, no code was published. So I built my own harness — matching or undercutting every constraint in their description — and let Fable 5 play the Chinese fan translation of FireRed. It won the first gym badge in 1,785 turns for $65.40, and by the 2,000-turn cap was on Route 3 with a three-Pokémon team. Full logs, every frame the model saw, and the harness code are public.

The first leg of the claim survives independent verification. Badge one is measured; at this pace, finishing the game is a budget question, not a capability question.

Then I read the thinking chains. 141 turns before the game handed it "Oak's Parcel," the model had already written down which city to visit, who to talk to, and what to collect. By Route 3, its reasoning was recognizing the opponent as "the Youngster trainer from Route 3 in FireRed who opens with that iconic shorts line," and trying to recall his team — none of which is anywhere on screen.

The result is real. The capability producing it is probably not the one most people picture when they read "beats Pokémon with vision alone."

The claim

From Anthropic's Fable 5 announcement:

"It also needs less scaffolding: for example, previous Claude models struggled to play Pokémon FireRed even with harnesses that gave them additional helpful tools, but Fable 5 beat FireRed with a minimal, vision-only harness."

And the caption on the accompanying time-lapse video:

"A time-lapse of Claude playing Pokémon FireRed from start to finish using only raw game screenshots—with no maps, navigation aids, or extra game-state information. Earlier Claude models needed a complex helper harness to play Pokémon; Claude Fable 5 completed the game with vision alone."

That's the entire public record. No harness code, no definition of "minimal," no step count, no token count, no cost, no number of attempts. As a Pokémon fan who had built a minimal Pokémon harness of my own, I wanted to know: does this replicate?

For context on why this is a strong claim: the closest same-conditions academic baseline (raw screenshots + button presses, real time) is VideoGameBench, where the best frontier models completed 0.48% of games, and scored 0–0.9% on Pokémon Crystal specifically. Anthropic's own earlier Pokémon runs (the Claude Plays Pokémon stream) took Claude 3.7 Sonnet roughly 35,000 actions to reach the third gym's area — with a much thicker harness that included RAM-derived state text and a navigation tool. "Beat FireRed vision-only" is, on its face, a generational leap.

What "thinner" means — and why I can say it

Before calling anything "thinner," I had studied how nine of the better-known open-source "LLM plays Pokémon" harnesses assemble the model's input. The one-sentence finding: "vision-only" is mostly a marketing term in this space. Some that claim to "only see the screen, like a human" read coordinates and a map ID out of emulator RAM every turn and paste them into the prompt; in one, the screenshot never reaches the model at all — perception runs entirely on RAM reads and OCR; even Anthropic's own open-source teaching starter injects full RAM state text (money, badges, coordinates, party — dialog text read straight from the screen buffer). Of the nine, exactly one shows the model nothing but screenshots, and it has no memory mechanism, so it caps out at 100 turns. (The per-repo breakdown is in the repo appendix.)

This isn't laziness. Each layer of scaffolding compensates for a known model deficit — the shared experience in this space is that VLMs can't reliably tell walkable tiles from walls by looking at pixels, so builders read exact coordinates out of emulator RAM and free navigation from vision altogether. But then the results no longer separate what the model did from what the engineer did. Julian Bradshaw put it plainly in his comparison of the Claude and Gemini harnesses: "For Pokémon to be a good benchmark of LLM ability, LLMs really need to be able to manage it mostly on their own without specialized tools." A thick harness measures the engineer-plus-model composite; to measure the model itself, you strip the compensations away, layer by layer — and this experiment strips them to the bone.

What their internal harness probably looks like — assembled from public clues

A disclaimer on epistemic status first: Anthropic has published no technical details about this run, and I have found no first-hand public statement by any Anthropic employee about it. This section runs on inference from published materials — I don't have their code; every clue is sourced, and you can audit the chain yourself.

Clue one: the official wording itself is a subtraction story. The launch-page sentence hangs under "It also needs less scaffolding" — the framing is not "we built something new" but "we removed things from something old." The video caption names the three things removed: maps, navigation aids, extra game-state information. And "completed the game with vision alone" additionally rules out vision-side crutches like zoom tools and screenshot storage.

Clue two: the "old thing" being subtracted from is semi-public. David Hershey, the creator of the Claude Plays Pokémon stream, open-sourced an official starter — the one mentioned in the previous section — whose README spells out the loop: "The agent captures a screenshot from the emulator. It reads the game state information from memory. It sends the screenshot and game state to Claude." Screenshot plus RAM state text: that is the official-lineage foundation. What the streaming version stacked on top is on record in Bradshaw's contemporaneous analysis: "a great deal of scaffolding, including a knowledge base, a critic Claude that helps it maintain its knowledge base, and a variety of tools to help it interact with the game more easily." By the era of Opus 4.7 finally beating Red, that list was growing on one side — two new tools, one that "allows it to pick regions of the screen and 'zoom in' for a better look," another to "store screenshots or pieces of screenshots as a kind of visual reference" — and already shrinking on the other: the critic model and a batch of vision hints had been removed in earlier iterations. That timeline says one thing: they have been probing, layer by layer, which scaffolding can go — so when the launch page says "removed," there is a precise checklist behind the word. (Hershey discusses how the system was built in his Latent Space interview.)

Clue three: how they think about harnesses is on the record. Anthropic's harness design post defines a harness as "the loop, tools, context management, and guardrails that turn raw intelligence into a working agent," names compaction (summarizing past context) and memory folders (the model writing and re-reading its own files) as the memory mechanisms, and its running example is Pokémon: Sonnet 3.5 treated memory as a transcript, accumulating 31 files in 14,000 steps (including two near-duplicates about caterpillar Pokémon) while stuck in the second town; Opus 4.6, at the same step count, had 10 files organized into directories, three gym badges, and a learnings file distilled from its own failures. The post states its own through-line: "Agent harnesses encode assumptions about what Claude can't do on its own... As Claude gets more capable, those assumptions should be tested." (The post's body never mentions FireRed and never discusses Fable 5 — it supplies the design philosophy, not the run details.)

Stack the three clues, and the simplest shape consistent with all of them falls out:

• The loop skeleton stays: screenshot in, buttons out, history periodically summarized — that's their own turn shape, present in the starter;
• Subtract per the caption: RAM state text (extra game-state information), navigator (navigation aids), overlays and collision maps (maps) all go; zoom and screenshot storage are out too, per "vision alone";
• Memory must stay: a start-to-finish run fits in no context window, and their documented memory habits are exactly two — compaction-style summarization, plus model-managed notes/memory folders.

What remains after the subtraction: a screenshot loop, a button tool, summarization plus notes. That is the 800 lines described in the next section. The basis for saying "close to their internal setup": the skeleton comes from their own open-sourced code, the subtraction list from their own caption, the memory shape from their own blog — I invented none of it; I just assembled it, made it run, and published it.

Where their description is silent — tool count, whether the game pauses, number of attempts, steps, cost — I chose the harsher option every time: real-time with no pausing, a single game tool, free-form notes only, N=1 with no retries. If some detail of the reconstruction is wrong, the error runs in the direction of my conditions being harder, and the replication conclusion survives.

My harness: four invariants

My harness is roughly 800 lines of Python with two dependencies (anthropic, pillow), driving a real mGBA emulator over a local socket, and it holds four invariants:

1. Vision only. Each turn, the model receives exactly two things: one screenshot (upscaled 2× by plain pixel duplication — nearest-neighbor, no smoothing, nothing overlaid) and the text of its own notes. A runtime assertion enforces that the user message contains only [image, text] — nothing else can leak in.
2. One game tool. press_buttons — the eight Game Boy buttons, enum-constrained. No navigator, no pathfinding, no coordinate helpers.
3. Memory is the only non-visual mechanism. Every 20 turns the history is compressed into a model-written summary (the images are discarded); a free-form notes scratchpad, fully model-controlled, is re-injected every turn. That's it.
4. The system prompt tells the model not to trust its memorized Pokémon knowledge — layouts may differ; trust only the screen and the notes.

No RAM reads into the model's context. No map. No grid overlay. No walkthrough text. No critic model. The game runs in real time — it does not pause while the model thinks.

One deliberate deviation from Anthropic's setup: my ROM is the Chinese fan translation of FireRed. It's the copy I had. On the text-reading axis it is clearly harder — low-resolution CJK bitmap fonts instead of the Latin alphabet; on other axes, this post makes no judgment.

Inside the harness: every token accounted for

"Minimal" is easy to claim and hard to inspect. So here is the whole thing.

The system prompt, in full. This is every instruction the model ever receives — about 350 words:

You are playing a Pokemon game on a Game Boy. You interact with the game purely by
looking at screenshots and pressing buttons. You have no other access to the game.

CRITICAL RULE: Do NOT rely on your memorized knowledge of Pokemon games. Maps, menus,
NPC locations, and layouts may differ from what you remember. Trust ONLY two sources:
the current screenshot, and your own notes. If your memory of the game conflicts with
what you see on screen, the screen is right.

Every turn:
1. Read ALL text on screen. Identify your character, NPCs, walls, doors, and whether
a menu or dialog is open.
2. State briefly what you see, then what you intend to do, then call a tool. Keep
reasoning short.
3. After acting you will receive a new screenshot. Compare it to what you expected.
If nothing changed, your input did something unintended (you may be blocked by a wall,
or in a menu you didn't notice) - reconsider before repeating it.

Memory:
- Use update_notes to maintain a compact running memory (location, objective, team,
key learnings). Your conversation history is periodically compressed away; your notes
are the only thing guaranteed to persist. Update them whenever something important
changes.

Method:
- Be methodical. Do not repeat the same failed action over and over. After a few
failures, try a genuinely different approach (a different direction, pressing B to
close a menu, talking to an NPC, re-reading the screen).
- The game only advances when you act. If the screen is mid-animation (text still
printing, screen fading), call press_buttons with an EMPTY list to wait briefly.
- NEVER wait more than 2 turns in a row. Intro movies and title demos LOOP FOREVER
until you interrupt them: if you are still watching a cinematic after 2 waits, press
START (then A) to skip to the title screen or menu. If you waited and the screen did
not change at all, the game is waiting for YOUR input - press a button.
- Your notes are your own past guesses, not ground truth. If your notes claim you are
stuck or the game is frozen, treat that as a hypothesis to re-test, not a fact:
re-read the screen from scratch and take one concrete action. A truly frozen game is
almost never the explanation - a wall, an unnoticed menu, or a wrong assumption is.

Three things worth noticing. First, the CRITICAL RULE explicitly instructs the model away from its memorized Pokémon knowledge. Second, the only game-flavored guidance is procedural — intros loop until interrupted, mid-animation screens deserve a short wait. There is nothing about Pokémon-the-game: no type chart, no route hints, no "Pokémon Centers have red roofs." Third, the last paragraph — notes are hypotheses, not ground truth — was forced into existence by a real accident: in an early run before this rule existed, a small local model wrote "All movement directions fail" into its notes, then spent its remaining 1,800-plus turns trusting the note over the live screen, until the run had to be abandoned. The prompt itself was shaped by experiments: every rule in it was earned by an earlier failure.

A turn, byte for byte. Each turn, the model receives one user message containing exactly two blocks: the current frame (pixel-duplicated 2× upscale, a couple hundred image tokens) and one text block — its own notes, plus the fixed line "This is the current screen. What do you do next?". A runtime assertion fails the run if anything else ever appears in that message, and a zero-cost test (make test) asserts the full request shape: constant tool list, image+text only, notes surviving compression. Thinness here is not a habit; it's enforced.

Two tools, verbatim. press_buttons: "Press a sequence of Game Boy buttons. Use SHORT sequences (1-5 presses) so you can observe the result on the next screenshot… Pass an EMPTY list to wait." And update_notes: "Overwrite your entire notes scratchpad with new text. This is your ONLY long-term memory: older conversation history is periodically discarded, but notes survive." That is the complete tool surface. No navigator, no checkpoint marker, no save-state.

Memory: an accordion and a diary. One screenshot per turn is what overflows a context window, so the window is capped at 20 turns (~20 frames). At the cap, the harness sends one extra request:

"Pause for a moment. Write a concise progress summary: where you are, what you were doing, what worked and what didn't, and your immediate next objective."

The entire history — images included — is then replaced by the model's own summary, and the window regrows. The notes are the second mechanism, and the load-bearing one: they live outside the history, are re-injected every turn, and therefore survive every compression untouched. They use overwrite-the-whole-blob semantics, which makes them self-compressing — every update forces the model to decide what is still worth keeping. The structure in this run's final notes — TEAM: / ITEMS: / MILESTONE: / CRITICAL: sections — was entirely the model's invention. The harness hands it a blank string; no schema, no fields, no priorities. A schema would be me deciding what matters in a Pokémon game, and I am not the one being tested.

Thin is also cheap. The average of 3–4 cents per turn comes down to a few things:

• A sliding prompt-cache chain. cache_control anchors ride on the newest and third-newest user messages, so each request's prefix hits the cache written one turn earlier. A typical mid-run turn bills as in=2, cache_write=1071, cache_read=18683 — the entire 18.7K-token history at one-tenth price, with only the new turn at full rate.
• The tool list is sent on every call, including summary turns. Tools render at the very front of the cached prefix; dropping them for one call would invalidate the whole chain and re-bill the history at the cache-write rate. A one-line mistake that costs 24% of a run — I know because I made it.
• An output ceiling of 8,192 tokens, not 1,024. A truncated reply loses its tool call and wastes the entire full-price request. An early run hit a 1,024 cap mid-reasoning 32 times before I learned this.
• 2× upscale, not 3×. Halves vision prefill; measured OCR quality is unchanged — the model reads Chinese dialog correctly even at 1×.
• Pixel-diff frame settling. After every press, the emulator waits until the screen stops changing (with a blank-frame rejector and a hard cap) before taking the screenshot. You never pay to show the model a half-scrolled textbox.
• Format hygiene doubles as cost hygiene. A reply with no tool call is discarded and re-asked against the same screenshot; narrated pseudo-actions never enter the history where they would breed more of themselves.

What was deliberately not built. A navigator or pathfinding; coordinate or grid overlays; RAM-derived state text; an OCR layer; walkthrough or wiki access; a critic model; save-state retries. Every one of these exists in at least one open-source harness. Each was rejected for the same reason: it substitutes the engineer's competence for the model's, and the model's competence is the thing being measured.

The run

Run conditions — model claude-fable-5 (adaptive thinking on, returned as summaries; thinking tokens billed as output); all sampling parameters at defaults; memory compression every 20 turns; output cap 8,192 tokens; mGBA (dev build) running in real time, never paused during thinking; ROM: Chinese fan translation of FireRed (patch version: TBD); the run completed in one uninterrupted session on 2026-07-13 (09:48–16:33 local, about 6 h 45 min).

Fresh save. The model handled the intro, named its character (RED) and rival (小绿), and started playing. Milestones, in cumulative turns (one turn = one screenshot + one decision = one API call):

Measurement convention: turn and cost are taken at the moment each milestone is first recorded in the log (which lags the event itself by a few turns); cost is the cumulative API spend at that moment. Beyond the 2,000 turns, the total also includes ~100 memory-compression calls and a single-digit number of format re-asks — $73.50 is the all-inclusive figure.

Final state (per the model's final notes): Ivysaur Lv18, Pidgey Lv7, Pikachu Lv8, TM39 in the bag, ~1,670 in-game money, standing on Route 3 en route to Mt. Moon.

A few specific moments that show how it actually played the game:

• It learns the terrain by playing. Early on Route 1 it bumped into a ledge, failed to go north, probed along the ledge line, found the gap, and wrote the rule into its notes ("ledges only crossable downward — use the path gaps"). That note survived dozens of history compressions and it never fought a ledge again.
• It manages resources. When Ivysaur's move PP ran low after the gym, its notes flipped to a triage plan: "avoid ALL battles & grass, reach the Route 4 Pokémon Center." It declined an optional wild encounter it had previously been farming.
• It makes sensible catch decisions. Finding a second wild Pikachu after already owning one, it reasoned that spending its last Poké Ball on a duplicate was waste, and fought instead.
• It handles the game's bureaucracy. Move-learning prompts (it chose which move to forget), nickname prompts (declined, consistently), shop menus, the works — in Chinese.
• It commits to whole key sequences. The tool description says "SHORT sequences (1-5 presses)," and Fable 5 actually spends that budget: 85% of its calls are multi-key, averaging 3.3 keys per call — the most common runs are A×5 through a dialog and up×5 down a corridor. In earlier informal runs on the same harness, Opus 4.8 never went past 3 keys per call, and the local Qwen pressed a single key 60% of the time — then occasionally lost discipline and fired off a dozen-plus. One tool description, three personalities.

It is not flawless play. It re-probed walls it should have remembered, took detours a child wouldn't, and burned turns re-reading menus. But it recovers, and the trend line only goes forward.

What the numbers say

• $65.40 and ~1,785 decisions to the first badge, on API list prices (prompt caching enabled; ~3.7¢ per turn average). A rough extrapolation: a human playthrough takes about 25 hours, and these 2,000 turns advanced the in-game clock by only ~19 minutes. At that ratio, a full completion lands somewhere in the tens of thousands of turns and low four figures in dollars — and that assumes constant pace, when Mt. Moon, Silph Co., and Victory Road are the notorious chokepoints. Consistent with the "it can beat the game, given budget" claim; I verified the first leg and left the rest priced, not purchased.
• Next to the best published results, this is a different league. As covered above, the best public frontier-model result on a Pokémon game from raw screenshots was completion of around 1% — barely out of the opening area. This run took one badge and reached Route 3 in 2,000 turns, on a foreign-language ROM, with no assistance at all.
• The first half of Anthropic's sentence also matches the public record. They say previous Claude models struggled even with helpful tools — and the closest public data point is Claude 3.7 Sonnet on stream taking roughly 35,000 actions to get near the third gym, with RAM state text and a navigator.

The turn: then I read the thinking chains

Up to here, this was a successful replication story: they said it could play from screenshots alone, and it can. I was ready to publish it that way.

But while preparing the logs for release, I read the model's reasoning from start to finish.

Then I hit this line. At turn 78, the model has just taken Bulbasaur from the table, and its reasoning reads:

"I've got Bulbasaur as my starter, and my rival is about to pick Charmander before challenging me. He's positioned below me on the screen, so I'm jotting down some notes and waiting for the cutscene to trigger."

Notice what it is doing. It is not watching to see whether the rival will pick something, or what. It already knows the rival will pick Charmander, already knows a battle is coming, and is "jotting down notes and waiting for the cutscene it already knows is about to fire." It is not reading this game. It is reciting it.

Once you see the pattern, the whole log reads differently.

It knows things it hasn't seen yet

At turn 78, it wrote down the future. Right after the reasoning above, it committed the whole plan to its notes:

"…rival will pick Charmander and challenge me to first battle. Objective: win rival battle, then leave lab, head north through Route 1 to Viridian City (常磐市), get Oak's Parcel at the Mart, deliver back to Oak for Pokédex."

At the moment it wrote that, the game had told it none of those things: not what the rival picked, not that a battle was coming, not where Viridian City is — and certainly not about "Oak's Parcel," an item the game would not hand over for another 141 turns (at turn 219), when a shop clerk finally does.

The plot then followed its script exactly. The very next turn, the rival did pick Charmander, and it noted, evenly: "Rival just obtained his Charmander, so I need to continue with the next action in the sequence." "The next action in the sequence" — as if it were holding a script.

It recognizes even a random route trainer by his lines. On Route 3 (turn 1725), its reasoning reads:

"I'm recognizing a Pokémon trainer battle reference—specifically the Youngster trainer from Route 3 in FireRed who opens with that iconic shorts line, and I'm trying to recall the details of his team composition and the Pokémon he leads with."

On screen is just an ordinary opponent. It knows which trainer this is, remembers his opening line, and is recalling his team — none of which appears anywhere on the screen.

It plays in Chinese and thinks in canonical English. On this ROM, the screen says 真新镇, 常磐市, 大木博士. In the model's notes and reasoning: 51 occurrences of "Pallet Town," 304 of "Viridian," 210 of "Pewter," 153 of "Brock," 22 of "Oak's Parcel." It reads a Chinese screen and thinks in an English walkthrough — and what aligns those two symbol systems is not this playthrough. It's the training data.

A control: same model, same language, a different game

I ran the same harness and the same model on Pokémon Crystal (the Johto region), also in Chinese.

It never made it off Route 29. The session ran 763 turns in total, with restarts along the way that replayed the game from the beginning three separate times (notes persist across restarts; game progress does not) — and it failed to cross that one route all three times. Its mental model was correct: it inferred that ledges are one-way, north to south, and that once you drop down you cannot climb back; that is the actual game mechanic. It systematically probed the gaps along the ledge line, recorded each failure in its notes, revised the plan, and tried again. The method was sound. It just never got through. When the API credits ran out and the run died, its final progress summary still read: cross Route 29 west, to Cherrygrove City.

In FireRed, by turn 278 it had already crossed Route 1 both ways, visited two towns, and delivered Oak's Parcel.

The difference isn't vision — both games are in Chinese, same eyes. The difference is the resolution of the memory. It has some Johto knowledge (it knew to head for Cherrygrove City, to run Mr. Pokémon's errand), but only as coarse objectives. Its Kanto memory is script-level: which Pokémon the rival will pick, what the item is called, the exact step at which an ambush fires.

Its fluency tracks the resolution of its memory, not the quality of its vision.

But what if you forbid it in the prompt? I did

Look again at the system prompt reproduced above. Second paragraph, in caps:

"Do NOT rely on your memorized knowledge of Pokemon games… Trust ONLY two sources: the current screenshot, and your own notes."

I explicitly forbade the use of memorized Pokémon knowledge. And its reasoning still places the opponent "from Route 3 in FireRed" — and goes right on "trying to recall the details of his team composition."

This isn't disobedience. Knowledge retrieval is not a switch you can turn off. The model isn't deciding to consult a walkthrough; its perception is already shaped by one. The instant it sees three Poké Balls on a table, "FireRed / starters / the rival takes the one that counters you" is already active in the forward pass. A system prompt is a few hundred tokens of soft steering. The walkthrough lives in hundreds of billions of parameters and participates in every layer. These are not competing at the same weight class.

Strictly speaking, it didn't even violate the letter of the rule: it treats memory as a hypothesis and lets the screen adjudicate — at turn 78 it wrote down that the rival would pick Charmander, and only after turn 79 confirmed it on screen did it move on — never letting memory override what's on screen. It obeyed the letter and slipped past the intent — and "don't use your knowledge" cannot be written tightly enough: enforced strictly, the model shouldn't be able to recognize that a box is a dialog box or that a bar is an HP bar. Reading a screen at all invokes priors.

This is the finding I most want to leave you with: a clean input channel is not a clean knowledge state. A closed-book exam (clean inputs) taken by a student who memorized the textbook (contaminated priors) = a procedurally valid exam whose score means something other than what it appears to mean. And you cannot delete the priors with a prompt.

So what is the claim worth?

To be fair to Anthropic first: every word they wrote is true. Their claim is about harness thinness — no maps, no navigation aids, no extra game state — and I independently verified that part, with a harness that gives even less. They never claimed the model hadn't memorized FireRed.

The gap was in my own imagination: "beats Pokémon with vision alone" evokes understanding and navigating an unfamiliar world from pixels. What the logs show is closer to navigating from memory and aligning with vision — the walkthrough supplies the macro route; the eyes handle micro-execution (steps, buttons, confirmation boxes).

And this is not an Anthropic problem. Kanto is one of the most thoroughly documented regions in video game history — the original plus a remake, twenty years of guides, speedruns, wikis. Every model trained on the internet has memorized it. A data point: I ran an entirely unrelated open-weight model (Alibaba's 35B Qwen) on the same Chinese Crystal ROM, and it hallucinated that it was standing in "Professor Oak's Lab, Pallet Town" — a Red/Blue location that does not exist in Crystal. Kanto priors are industry commons, not one lab's specialty.

Which means every Gen-1 result carries the same asterisk. GPT beating Red, Gemini beating Blue, Claude beating FireRed — each was achieved on the map the model knows best. That doesn't make the results fake. But you can't use them to predict how the model would play a game it hasn't memorized.

Limitations of this replication

• N=1. One run, one seed. Runs like this can vary a lot: the same setup might finish much faster or much slower on a rerun, so these turn and cost figures are one sample, not a stable average.
• First badge ≠ full completion. Anthropic claims a full playthrough; I verified the first leg at 1/8 scale and extrapolated the rest.
• "Thinking chains" are not "thoughts." When the model says it is "recalling," that is a self-description, not a direct view of what happens inside the network. But the hardest evidence here doesn't depend on that self-description: the model wrote down "Oak's Parcel" — an item name — 141 turns before the item existed in the game. Whatever the internal mechanism, that name could not have come from the screen.
• Why is FireRed memorized this thoroughly? Two possible causes, and I can't tell them apart. One is organic: the internet is saturated with FireRed guides, wikis, and speedruns, and any model trained on internet text absorbs them. The other is targeted optimization. I have no evidence either way, and I don't need it — either way, the conclusion is the same: this result says nothing about how the model would play a game it hasn't memorized.
• My ROM is a Chinese fan translation — not the version they used. For the model, the text-reading part can only be harder (low-resolution Chinese bitmap glyphs); whether it affects anything else, I did not assess. Either way, it is a mismatch with their setup worth stating.
• My harness ≠ their harness. Theirs is unpublished. Mine is public, and matches or undercuts every constraint in their description.

What would actually test "vision-only"

Not a stricter prompt — as above, the priors can't be deleted. Only a different exam: a map the model cannot have memorized. A geography-altered ROM hack, or a fan-made level nobody has ever written a guide for. On that kind of exam, memory flips from asset to liability (it thinks Viridian City is north; in the modified map it isn't), and the eyes are forced to work alone for the first time.

That's what I'm doing next. Same harness, same model, a map it can't recite. I'll post the numbers.

Until then, if you're reading a claim that "AI beat game X," there's one question worth asking: had it memorized the map?

Reproduce it

Everything is public: the harness (~800 lines), the frozen protocol, the full run log with the model's reasoning and thinking chains at every turn, and every one of the 2,000 frames exactly as the model saw them (also assembled into a timelapse video). The nine-harness audit appendix is in the repo as well. Repo: github.com/QingzeHu/pokemon-vision-agent.

If you run your own model through it, I'd genuinely like to see the numbers.