$ account NanmiCoder
MediaCrawler is a multi-platform social media data collection tool targeting Chinese platforms: Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Baidu Tieba, and Zhihu. It uses Playwright browser automation to log in (QR code, phone, cookie) and then scrapes posts, comments, and media files based on search keywords, specific post IDs, or creator IDs. The architecture is a monolith with modular platform-specific crawlers, a shared abstract base layer, and a FastAPI backend that provides a WebUI for visual control. Data can be stored in multiple formats (CSV, JSON, JSONL, SQLite, MySQL, PostgreSQL, MongoDB, Excel). Additional features include a robust media downloader with DASH support, proxy rotation, word cloud generation, and privacy-focused anonymization. The codebase demonstrates solid engineering with design patterns (factory, abstract base, mixin), good test coverage, and a clean separation of concerns, though there is some duplication across platform implementations.