$ show evan-kolberg/prediction-market-backtesting
This repository implements a comprehensive backtesting and live-trading framework for prediction markets (specifically Polymarket and Kalshi), built on a heavily customized NautilusTrader engine. It provides historical replay of L2 order book data from multiple vendors (Telonex, PMXT, Polymarket native), realistic execution modeling (fill models, fee models with maker rebate inference), and a library of trading strategies (e.g., EMA crossover, microprice imbalance, late favorite hold). The framework supports parameter search optimization (random grid, TPE via optuna), joint-portfolio multi-market backtests, and generates detailed HTML/CSV/JSON reports with equity curves, drawdown, and Brier advantage analysis. A Rust native extension accelerates data conversion (book diffing, payload sorting), and a CLI TUI (main.py) discovers and runs backtest entrypoints. While primarily an application framework, it also includes numerous one-off research scripts and Jupyter notebooks (e.g., BTC snapshot model walkforward), giving it a hybrid character: a production-quality backtesting infrastructure with a heavy research emphasis. The codebase is well-tested (50+ test files), documented, and follows clean engineering practices (config validation, structured event logging, concurrency patterns). However, it is tightly coupled to NautilusTrader internals, relies on several environment variables, and contains large monolithic loader classes (e.g., Telonex loader at ~3600 lines) that limit reusability outside its specific domain. Maintenance appears active (last commit 2026-05-16) with low technical drift, though the heavy dependency on NautilusTrader and optional Rust build are potential friction points.
The LLM identified 3 files worth highlighting for reuse:
Provides reusable field validators (require_probability, require_positive_decimal, require_between) that are intended to be used in dataclass __post_init__ methods. This isolates validation logic from strategy config classes, making it easy to enforce constraints on configuration fields across many strategy configs. The pattern can be extracted into a separate library for any Python dataclass-based configuration system.
Implements a structured event logging system for data loading progress, with pluggable sinks (console progress bar, JSONL file, in-memory capture). Events are typed dataclasses (LoaderEvent) with vendor, stage, and status fields. This pattern is generic enough to be used in any multi-stage ETL pipeline to track and expose progress without coupling to a specific logging framework.
Defines PredictionMarketTakerFillModel, a configurable fill model for prediction market backtests that supports tick-based and percentage-based slippage, as well as a configurable probability price floor. This class can be extracted and adapted to any prediction market simulator, providing realistic execution modeling without relying on the full NautilusTrader execution stack.