Claude plays GBA games autonomously via MCP. Not simulated. Not cheating. Actually playing — vision, reasoning, button presses — frame by frame.
How it works
Claude sees every frame via MCP. It dispatches to specialized sub-agents, presses buttons, and writes persistent game knowledge — all in real time, all local.
6 specialized sub-agents
Claude dispatches to the right specialist for each situation — reading pixels, planning routes, countering opponents, recording knowledge.
Philosophy
Most "AI plays games" projects are scripts. WhyEmuGBA is not. Claude sees the screen and decides — every turn, every frame, every routing decision comes from actual reasoning over actual visual input.
The same Claude that writes code and answers questions is now pressing A and B. No separate model. No special training. Just vision, reasoning, and tool use — applied to a Game Boy Advance cartridge.
Different problems require different expertise. Six sub-agents with defined roles, activated dynamically — the right specialist handles each situation.
memory.json grows with every session. Encounter patterns, route knowledge, battle outcomes — all persisted. Claude plays better the second time through any area.
The 120-frame buffer means bad decisions are never permanent. Claude can rewind, reconsider, and retry — building better strategies through iteration.
No cloud. No API key for the emulator side. The Node.js server, the browser, the memory, the sessions — all local. Your game data stays on your machine.
Under the hood
Every call to get_screen captures the current mGBA canvas and returns it to Claude for analysis. No frame is missed.
press_button, skip_frames, and ai_control drive the emulator directly via WebSocket. Claude inputs exactly what a human would.
Cross-session memory.json stores encounter patterns, route maps, battle data. The auto-improvement system detects and proposes UX enhancements.
IndexedDB on the browser side, filesystem sessions on the server. Full log of every action, chat, thought, and screenshot — per game, per session.
6 prebuilt specialists plus create_agent for runtime agents. Each has a defined role, tools, and activation criteria. Orchestrated by Claude.
Sidebar (ROM info, agent status), game canvas with controls, chat panel. Floating draggable emulator window. Gamepad, keyboard, touch, and drag-and-drop ROM loading.
20 MCP tools
Every action Claude can take — from reading pixels to pressing buttons to managing sub-agents.
// Memory
memory.json grows with every session — route patterns, battle outcomes, encounter rates, optimal paths. Ask Claude to recall what it learned about Route 1 three sessions ago. It knows.
The more games Claude plays, the richer the knowledge base. That memory is yours — persistent, local, and impossible to replicate by starting over.
Most "AI plays games" demos are scripts.
This one actually thinks.
WhyEmuGBA is for anyone who wants to see what happens when you give a general-purpose reasoning model full sensory access to a game — no fine-tuning, no pre-programmed moves, no cheating. Just vision, tool use, and persistent memory applied to a GBA cartridge.
The more sessions Claude plays, the better it gets. Route knowledge compounds. Battle patterns accumulate. Mistakes get rewound and corrected. It doesn't just play the game — it learns to play it.
The gap between AI assistant and AI player — we're closing it, one frame at a time.
Get started
Node.js MCP server, mGBA in the browser, Claude Code as the AI brain. No cloud dependency. Drop a ROM, connect via MCP, and Claude starts playing immediately.