This is a research-grade PyTorch implementation of Danijar Hafner's Dreamer v4 model-based reinforcement learning algorithm. The core artifact is the `dreamer4` package, dominated by a monolithic `dreamer4/dreamer4.py` (~3000 lines) that implements `VideoTokenizer`/`SpaceTimeTokenizer`, `DynamicsWorldModel`, `AudioTokenizer`, and `StateTokenizer`. These classes build causal space-time transformers with separate space/time attention blocks, latent flow-matching diffusion with shortcut consistency, BYOL-style tokenizer pre-training, PoPE attention, and a large zoo of experimental loss terms (latent autoregressive loss, decorrelation, orthogonality, SigReg, self-flow distillation, shortcut loss). `dreamer4/trainers.py` provides training loops (VideoTokenizerTrainer, BehaviorCloneTrainer, DreamTrainer, SimTrainer) and `dreamer4/cli.py` exposes CLI entry points for training and serving the world model. The repository is hybrid: primarily research code, but packaged and tested like a library. It ships a pyproject.toml with pinned dependencies, a parametrized test suite (tests/test_dreamer.py, test_e2e.py, test_env_wrapper.py), a browser-based Snake environment with a Flask-style server (`dreamer4/web_env/`) used for human demonstration collection and replay-buffer inspection, and environment wrappers in `dreamer4/env.py` that record agent experience to folders or memmap replay buffers. Root-level scripts are the actual experiment entry points, targeting standard benchmarks: HalfCheetah (imagination RL), CartPole (dynamics + RL), MovingMNIST (tokenizer + dynamics), and Snake (PPO with dynamics model). Craftsmanship is above typical research code: clean package layout, pytest coverage across many hyperparameter combinations, self-contained utilities (LossNormalizer, tensor pack/unpack helpers, with_seed decorator, memmap replay-buffer helpers), and HuggingFace Accelerate integration for distributed/mixed-precision training. However, the single-file core, 100+ hyperparameter flags on trainers, scattered hard-coded constants, sparse docstrings, and reliance on volatile niche packages (x-transformers, h-net-dynamic-chunking, PoPE_pytorch, adam-atan2-pytorch) make long-term stability and code reuse risky. The repository is recently committed and appears maintained, but it is explicitly beta-status experimental research code, not a production system.
FinRL-Meta is a comprehensive Python library and framework for applying deep reinforcement learning (DRL) to financial trading. It provides a modular, plug-and-play ecosystem consisting of market environments (stock, crypto, futures, forex, portfolio allocation, order execution optimization, market impact), data processors (Yahoo Finance, Alpaca, Binance, Tushare, etc.), and DRL agent wrappers (Stable-Baselines3, ElegantRL, RLlib). The library implements the full training-testing-trading pipeline, with CLI entry points (train.py, test.py, trade.py), backtesting engines (WonderTrader integration for futures), and paper trading capabilities (Alpaca). It is the market environment layer of the larger FinRL ecosystem and includes research-grade features such as market impact models, turbulence index, and ensemble strategies. The codebase is hybrid: while primarily an application/library, it contains numerous Jupyter notebooks, experimental sub-projects (order execution with qlib, high-frequency futures), and academic paper references. This reflects its dual role as both a reusable software framework and a research reproducibility platform. The architecture is well-structured with abstract base classes (data processors, impact models, environments), factory patterns, and consistent Gym/Gymnasium interfaces. However, the repository also carries technical debt from its research origins: hardcoded paths, mixed gym versions, deprecated library usage, and some incomplete or commented-out code. The most reusable components are the data processor base class (providing a common data pipeline) and the pluggable impact model hierarchy, both of which can be extracted with minimal modification for other financial RL or simulation projects.
This repository hosts the Hugging Face Deep Reinforcement Learning Course (v2.0), a free, open-source educational curriculum covering RL from fundamentals (Q-Learning, Policy Gradients) to advanced topics (PPO, A2C, Multi-Agent RL, Curiosity-Driven RL). It is primarily a collection of theoretical lessons (MDX files) and hands-on Jupyter notebooks designed to run on Google Colab. The notebooks train agents using libraries like Stable-Baselines3, CleanRL, ML-Agents, and Sample Factory, and push trained models to the Hugging Face Hub with evaluation videos and leaderboard comparison. While not a production application or data pipeline, the repo contains reusable training loops, evaluation helpers, and Hub packaging functions that can be extracted for other RL projects. The course is in low-maintenance mode, with some broken features (Unit 7 AI vs AI) and dependency drift (gym vs gymnasium), but the core content remains functional and well-structured.
This repository contains educational labs for MIT's Introduction to Deep Learning course (6.S191). It consists of Jupyter notebooks covering deep learning fundamentals (Lab1: music generation with RNNs), computer vision (Lab2: MNIST classification and facial detection with VAEs), and LLM fine-tuning (Lab3: style adaptation using LoRA). The repository also includes extra labs for reinforcement learning and uncertainty quantification. While structured as educational material, the labs involve substantive data processing, model training, and evaluation pipelines. The mitdeeplearning package provides reusable utilities for training visualization and data handling. The code assumes execution in Google Colab with GPU access and relies on external APIs (Comet ML, OpenRouter) and datasets hosted on third-party services (Dropbox).