$ show PlummersSoftwareLLC/ArcadeAI
This repository implements a real-time reinforcement learning training system for two classic arcade games: Robotron: 2084 and Tempest. The system uses MAME emulators with Lua scripts to extract game state and apply actions, communicating via TCP sockets to Python agents. The Python side includes multiple RL algorithms: a Rainbow DQN (distributional C51 with factored action heads) for Tempest, and both a DQN and a PPO (Set Transformer) variant for Robotron. Training is supported by prioritized experience replay, n-step returns, behavioral cloning from heuristic experts, and a live metrics dashboard with real-time charts and game preview. The repository also contains the original assembly source code for both games, included for reference but not actively used by the training pipeline. The codebase is functional and well-structured in parts, but suffers from duplication between the two game projects, legacy code in an '_old_engine' directory, and platform-specific dependencies. It is actively maintained with recent training configuration tweaks documented in notes.
The LLM identified 3 files worth highlighting for reuse:
Implements PrioritizedReplayBuffer with a sum-tree data structure for O(log N) proportional sampling. Supports vectorized batch operations, memmap-based storage for large buffers, and priority boosting. The SumTree class is self-contained and can be reused in any RL project requiring prioritized replay. The buffer is decoupled from game-specific logic and is used by both Tempest and Robotron agents.
Provides NStepReplayBuffer, a generic sliding-window accumulator for n-step return calculations. It handles actor boundaries by splitting episodes, and outputs pre-assembled transitions ready for training. This component is imported by the socket server and is reusable in any RL setup that requires multi-step returns, independent of the specific game environment.
A self-contained Grafana-style metrics dashboard served via Python's ThreadingHTTPServer. It collects rolling window statistics (rewards, loss, FPS, Q-values) and serves a single-page HTML dashboard with gauges, sparklines, and a game preview. The dashboard is launched from the main training script and can be adapted to any RL training pipeline with minimal changes. It includes WebRTC audio/video streaming support, system resource monitoring, and a keyboard handler for interactive control.