ArcadeAI is an active reinforcement-learning research codebase that trains deep RL agents to play Robotron: 2084 and Tempest inside MAME. The actual production loop works as follows: Lua scripts running inside MAME instances read game memory, flatten state into binary packets, and send them over TCP to Python servers. The Python side parses frames, computes rewards, processes states into entity/ray/grid tensors, runs DQN or PPO policies, and returns encoded joystick actions back to the emulator. Training uses Rainbow-style DQN for Tempest and a newer PPO implementation for Robotron, with expert rule bootstrapping, behavioral-cloning losses, prioritized experience replay, n-step returns, guidance-rescue mechanisms, and heavily configurable reward shaping. Both game projects include live Grafana-style web dashboards for metrics, game preview, and system monitoring. Architecturally, the repository is split into MAME-side Lua clients, socket bridges, state processors, model definitions, replay buffers, async inference batchers, and training utilities. This is a hybrid research/application repository: it is not a packaged library or service, but a set of tightly coupled research pipelines with duplicated infrastructure across the Tempest and Robotron directories. The repository also contains original Williams and Atari 6809/6502 assembly sources and ROM staging scripts; these appear to support the RL work by documenting game memory layouts, but they are not part of the active training runtime. The strongest assets are genuinely reusable: a sum-tree based prioritized replay buffer, a generic n-step return buffer, and a configurable reward-shaping module with potential-based movement shaping. These components are well-encapsulated and could be extracted into new RL projects. However, the overall codebase is a personal research dump: there are multiple abandoned engines (_old_engine), hardcoded hostnames and environment-specific paths, 3000+ line Lua/Python scripts, duplicate code across game directories, and tight coupling to MAME memory addresses. It is actively maintained and shows high engineering effort in the core training infrastructure, but reuse outside this repository would require selective extraction rather than direct adoption.
This repository is the companion code and documentation for the JoyRL Book, a Chinese-language reinforcement learning textbook. It contains over 30 self-contained Jupyter notebooks implementing classic and modern RL algorithms (DQN, Double DQN, Dueling DQN, Noisy DQN, PER DQN, Categorical DQN, Rainbow DQN, REINFORCE, A2C, PPO, DDPG, TD3, SAC, TRPO, ACER, CQL, etc.) on Gymnasium environments like CartPole-v1, Pendulum-v1, and Acrobot-v1. The site is built with Docusaurus and includes interactive playgrounds (e.g., DQN Playground), visitor tracking via a Cloudflare Worker, and extensive pseudocode in LaTeX. The code is educational: each notebook is self-contained with model, replay buffer, policy, training loop, and plotting utilities. There is heavy code duplication across notebooks, no shared library, and no unit tests, but the implementations are correct and well-documented for learning purposes. The repository also includes documentation on offline RL and LLM+RL (placeholders).