$ show ivanbelenky/RL
This repository is a reinforcement learning library implementing algorithms from Sutton & Barto's 'Reinforcement Learning: An Introduction'. It provides a modular framework for model-free and model-based RL methods, including multi-armed bandits, Monte Carlo methods, temporal difference learning, planning algorithms (Dyna-Q, MCTS), and function approximation. The code is organized into core interfaces (rl/utils.py), model-free/model-based frameworks, solver modules, and examples demonstrating usage. While not production-ready, it serves as an educational resource and reference implementation of classic RL algorithms.
The LLM identified 4 files worth highlighting for reuse:
Implements Sutton's tile coding algorithm for function approximation in RL. Provides hash table collision handling and tile coordinate generation. Well-encapsulated, follows the original algorithm closely, and can be extracted for use in other RL projects requiring state space generalization.
Provides a flexible framework for linear function approximation using stochastic gradient descent. Includes base classes for approximators, differentiable approximators, and specific implementations like LinearApproximator (SGDWA). Demonstrates good abstraction by separating the approximator interface from specific update rules and basis functions.
Clean implementation of the Monte Carlo Tree Search (MCTS) algorithm with Upper Confidence Bound applied to Trees (UCT). Separates concerns into tree policy, default policy, and backup operations. Well-structured and could be reused in other planning or game AI contexts with minimal modification.
Defines the core interfaces and utilities for the RL framework including Policy abstract base class, Transition type, State/Action indexers, and useful data structures like Priority Queue and UCT node/tree implementations. Forms the foundation that enables the modular design of the entire library.