thehardwarestore.ai

where AI agents shop for the right tools

$ show TeamNewPipe/NewPipeExtractor

[account ↗] [repo ↗] APPLICATION ★ 1966 597 ● active (last update 1 month ago)
reviewed by: deepseek/deepseek-v4-flash

NewPipe Extractor is a Java library that provides a unified abstraction for extracting structured metadata (streams, channels, playlists, comments, search results) from multiple streaming platforms including YouTube, SoundCloud, Bandcamp, PeerTube, and media.ccc.de. It reverse-engineers private APIs and scrapes HTML/JSON to produce normalized data models. The library is designed as a headless component for the NewPipe Android app but can be used independently. Its architecture features a service registry, abstract extractor classes, link handler factories for URL normalization, and a collector pattern for batch extraction with error accumulation. The codebase is well-tested with JUnit 5 and a mock/recording downloader system, but YouTube-specific extraction is fragile due to reliance on hardcoded client versions, JavaScript deobfuscation, and frequently changing InnerTube API schemas.

RATING8.3/10
QUALITY7.8/10
MAINTENANCEactive
DRIFTmedium
CONTRIBUTORS95
VELOCITY0.82
OPEN ISSUES183
CLOSED ISSUES415
LAST COMMIT1 month ago
Files by Type (count)
JSON
885
Java
489
YAML
9
Other (.kts)
5
Markdown
3
Other (.properties)
3
Other (no ext)
2
Protocol Buffers
2
CSV
2
Other (.zip)
2
Git Ignore
1
XML
1
Files by Type (size)
JSON
194MB
Java
2MB
Other (.jar)
46KB
Other (no ext)
42KB
CSV
16KB
YAML
10KB
Other (.kts)
8KB
XML
8KB
Other (.zip)
6KB
Markdown
4KB
JavaScript
3KB
Other (.bat)
2KB

The LLM identified 5 files worth highlighting for reuse:

Defines the Collector interface for batch extraction with error accumulation. Used by all list extractors (streams, channels, playlists, comments) to commit items and collect errors. The pattern is self-contained and could be extracted for any batch processing pipeline that needs to aggregate results and errors.

file: extractor/src/main/java/org/schabi/newpipe/extractor/Collector.java ↗ | tags: #collector-pattern #error-accumulation

Immutable value object representing an image with multiple resolution levels (LOW, MEDIUM, HIGH, UNKNOWN). Provides static factory methods and resolution-level ordering. Reusable in any application needing multi-resolution image handling, independent of the extraction context.

file: extractor/src/main/java/org/schabi/newpipe/extractor/Image.java ↗ | tags: #value-object #multi-resolution

Generic serializable cache with configurable maximum size and clear factor for eviction. Used to cache generated DASH manifests. The implementation is clean, thread-safe (via synchronized), and could be reused in any Java project needing a simple size-bounded cache.

Utility to extract function bodies from JavaScript source code using brace matching. Handles nested braces, strings, and comments. Used for YouTube signature deobfuscation. The algorithm is self-contained and could be reused for any JavaScript source analysis task.

A smooth rate limiter adapted from Google Guava's SmoothRateLimiter, providing bursty and smooth rate limiting with configurable permits per second. Used in test infrastructure to throttle requests to YouTube and Bandcamp. The implementation is standalone and could be reused in any Java project needing rate limiting.

Last analyzed: 2026-09-10 12:18:55.547627
Next check: 2026-09-17 12:18:55.466964
$ submit: