Active · MCP Server :9876 EXPERIMENTAL

Claude plays GBA.
Frame by frame.

Not simulated. Actually playing.

Claude plays GBA games autonomously via MCP. Not simulated. Not cheating. Actually playing — vision, reasoning, button presses — frame by frame.

FRAME 0001 analyzing game state…
SELECT START
B
A
[ claude ] reading frame 0001…
// Node.js · TypeScript · mGBA WASM · WebSocket · 20 MCP TOOLS Vision · Battle · Explorer · Text · Controller · Memory SCROLL
WhyEmuGBA in action

Three layers.
One AI brain.

Claude sees every frame via MCP. It dispatches to specialized sub-agents, presses buttons, and writes persistent game knowledge — all in real time, all local.

01 // CLAUDE AI Claude Code MCP / stdio get_screen tool call frame data response button press 02 // MCP SERVER Node.js :9876 20 tools · 6 sub-agents memory.json sessions/ WebSocket bridge · REST API · auto-save 30s 03 // BROWSER mGBA WASM Vision Agent Battle Agent Explorer Memory Agent
20MCP tools available
6specialized sub-agents
120frame rewind buffer
100%local — no cloud required

6 specialized sub-agents

Each agent is an expert.
Activated dynamically.

Claude dispatches to the right specialist for each situation — reading pixels, planning routes, countering opponents, recording knowledge.

Vision Agent
get_screen → analyze frame → report game state
Reads every pixel of every frame. Identifies HP bars, map coordinates, enemy positions, menu states, dialogue text — anything visible on screen.
Battle Agent
Wild Rattata detected → analyze type matchup → select TACKLE
Analyzes the opponent's type, level, and moves. Selects the optimal attack, manages PP, decides when to use items or switch. Battle logic, not luck.
Explorer Agent
Path blocked at Route 3 → reroute → waypoint: Mt. Moon
Builds and maintains route maps across sessions. Remembers blocked paths, ledge shortcuts, hidden items. Gets smarter with every new area explored.
Memory Agent
Route 1 pattern saved — 4 Rattata encounters recorded
Records every encounter, move outcome, and map event to memory.json. The more sessions played, the richer the knowledge base. It compounds.
120-Frame Rewind
Bad move on turn 3 → rewind 60 frames → retry with Ember
Every mistake is recoverable. The rewind buffer holds 120 states — Claude can step back, reconsider, and try a different approach without losing progress.
Voice Control
"Vai al Pokemon Center" → TTS response → navigation begins
Give Claude instructions in Italian while it plays. Voice input via SpeechRecognition, TTS responses via the macOS say command. Hands-free direction.

Philosophy

Built around one conviction:
real intelligence, real play.

01 // NOT SCRIPTED

No save state exploitation. No pre-programmed moves.

Most "AI plays games" projects are scripts. WhyEmuGBA is not. Claude sees the screen and decides — every turn, every frame, every routing decision comes from actual reasoning over actual visual input.


The same Claude that writes code and answers questions is now pressing A and B. No separate model. No special training. Just vision, reasoning, and tool use — applied to a Game Boy Advance cartridge.

02 // MULTI-AGENT

Six specialists. One brain.

Different problems require different expertise. Six sub-agents with defined roles, activated dynamically — the right specialist handles each situation.

03 // SELF-IMPROVING

Gets better every session.

memory.json grows with every session. Encounter patterns, route knowledge, battle outcomes — all persisted. Claude plays better the second time through any area.

04 // REWINDABLE

Every mistake is recoverable.

The 120-frame buffer means bad decisions are never permanent. Claude can rewind, reconsider, and retry — building better strategies through iteration.

05 // FULLY LOCAL

Everything runs on your Mac.

No cloud. No API key for the emulator side. The Node.js server, the browser, the memory, the sessions — all local. Your game data stays on your machine.


Under the hood

Real engineering.
Real numbers.

01 // VISION

Frame-by-frame screen capture

Every call to get_screen captures the current mGBA canvas and returns it to Claude for analysis. No frame is missed.

02 // INPUT

Real button presses

press_button, skip_frames, and ai_control drive the emulator directly via WebSocket. Claude inputs exactly what a human would.

03 // MEMORY

Persistent game knowledge

Cross-session memory.json stores encounter patterns, route maps, battle data. The auto-improvement system detects and proposes UX enhancements.

04 // SESSIONS

Auto-save every 30 seconds

IndexedDB on the browser side, filesystem sessions on the server. Full log of every action, chat, thought, and screenshot — per game, per session.

05 // AGENTS

Dynamic sub-agent system

6 prebuilt specialists plus create_agent for runtime agents. Each has a defined role, tools, and activation criteria. Orchestrated by Claude.

06 // INTERFACE

3-column browser UI

Sidebar (ROM info, agent status), game canvas with controls, chat panel. Floating draggable emulator window. Gamepad, keyboard, touch, and drag-and-drop ROM loading.


20 MCP tools

Every action Claude can take — from reading pixels to pressing buttons to managing sub-agents.

get_screen press_button skip_frames get_game_info save_state load_state chat_message think ai_control activate_agent deactivate_agent create_agent get_memory decision tts get_session_info get_improvements apply_improvement log_action get_agents_status get_screen press_button skip_frames get_game_info save_state load_state chat_message think ai_control activate_agent deactivate_agent create_agent get_memory decision tts get_session_info get_improvements apply_improvement log_action get_agents_status

Every encounter.
Remembered forever.

memory.json grows with every session — route patterns, battle outcomes, encounter rates, optimal paths. Ask Claude to recall what it learned about Route 1 three sessions ago. It knows.

The more games Claude plays, the richer the knowledge base. That memory is yours — persistent, local, and impossible to replicate by starting over.

memory.json · session #12
GAME: Pokemon Red · Route 1 12 sessions total
09:14:22VISION Pokémon Red, Route 1, HP 34/45
09:14:23BATTLE Wild Rattata detected, move TACKLE selected
09:14:25MEMORY Route 1 pattern saved — 4 Rattata encounters
09:14:28EXPLORER Path optimized, next waypoint: Viridian City
09:14:30MEMORY Auto-save written · IndexedDB + sessions/
memory.json — game knowledge 68%
session history12 sessions
// Mission

Most "AI plays games" demos are scripts.
This one actually thinks.

WhyEmuGBA is for anyone who wants to see what happens when you give a general-purpose reasoning model full sensory access to a game — no fine-tuning, no pre-programmed moves, no cheating. Just vision, tool use, and persistent memory applied to a GBA cartridge.

The more sessions Claude plays, the better it gets. Route knowledge compounds. Battle patterns accumulate. Mistakes get rewound and corrected. It doesn't just play the game — it learns to play it.

The gap between AI assistant and AI player — we're closing it, one frame at a time.

Load a ROM.
Watch Claude play.

Node.js MCP server, mGBA in the browser, Claude Code as the AI brain. No cloud dependency. Drop a ROM, connect via MCP, and Claude starts playing immediately.

20+
MCP tools — full AI game control
6
specialized sub-agents activated dynamically
120
frame rewind buffer — every mistake recoverable
100%
local — no cloud, no API key for the emulator side