$ show huggingface/deep-rl-class
This repository hosts the Hugging Face Deep Reinforcement Learning Course (v2.0), a free, open-source educational curriculum covering RL from fundamentals (Q-Learning, Policy Gradients) to advanced topics (PPO, A2C, Multi-Agent RL, Curiosity-Driven RL). It is primarily a collection of theoretical lessons (MDX files) and hands-on Jupyter notebooks designed to run on Google Colab. The notebooks train agents using libraries like Stable-Baselines3, CleanRL, ML-Agents, and Sample Factory, and push trained models to the Hugging Face Hub with evaluation videos and leaderboard comparison. While not a production application or data pipeline, the repo contains reusable training loops, evaluation helpers, and Hub packaging functions that can be extracted for other RL projects. The course is in low-maintenance mode, with some broken features (Unit 7 AI vs AI) and dependency drift (gym vs gymnasium), but the core content remains functional and well-structured.
The LLM identified 4 files worth highlighting for reuse:
Implements tabular Q-Learning from scratch for FrozenLake-v1 and Taxi-v3. Contains a reusable `train` function with epsilon decay, Bellman update, and evaluation. Also includes `push_to_hub` and `load_from_hub` functions that package the Q-table, evaluation results, and a replay video into a Hugging Face Hub model repository. This pattern is self-contained and can be adapted to any discrete-action RL environment.
Implements the REINFORCE (policy gradient) algorithm from scratch for CartPole-v1 and PixelCopter. Defines a `Policy` neural network class, a `reinforce` training loop with return-to-go and return standardization, and an `evaluate_agent` function. The Hub packaging logic (`push_to_hub`) is reusable for any policy-based RL agent. Demonstrates clean separation of model definition, training, evaluation, and deployment.
Contains a full PPO implementation from scratch (based on CleanRL) with actor-critic networks, GAE advantage estimation, minibatch updates, and KL early stopping. The `package_to_hub` function bundles the model, evaluation metrics, and replay video for Hub upload. This is a production-quality PPO training script that can be extracted and used as a standalone training pipeline for any Gymnasium environment.
Demonstrates training a PPO agent using Stable-Baselines3 on LunarLander-v2. Uses the `huggingface_sb3` library to push the model to the Hub with automatic evaluation and replay video generation. The pattern of using `package_to_hub` from `huggingface_sb3` is a reusable template for any SB3-trained agent.