$ show samtools/htsjdk
htsjdk is a comprehensive Java library for reading, writing, and manipulating high-throughput sequencing (HTS) data formats, including SAM, BAM, CRAM, VCF, BCF, FASTQ, FASTA, BED, and GFF3. It provides format-specific codecs, indexing (BAI, CRAI, CSI, Tabix, tribble), block-compressed I/O (BGZF with libdeflate acceleration), and a plugin-based codec registry (beta package). The library is used by major bioinformatics tools like Picard, GATK, and IGV. It includes extensive CRAM compression support (rANS, FQZComp, range coding, name tokenisation) and versioned codec implementations for CRAM 3.0/3.1 and VCF 4.0-4.3. The architecture follows a layered design: low-level I/O streams, format-specific readers/writers, and a higher-level plugin framework for content-type decoupling. While the codebase is actively maintained and well-tested, it contains some deprecated APIs, experimental beta code, and a few unimplemented code paths, reflecting its long evolution and breadth of format support.
The LLM identified 6 files worth highlighting for reuse:
Abstract base class for random-access streams, with implementations for file, HTTP, FTP, memory, and byte-array sources. Provides position(), seek(), read() methods and is used as the foundation for all remote/local file access in htsjdk. The pattern is cleanly extensible and could be extracted as a standalone streaming abstraction for any project needing seekable byte input.
Implements BGZF (Block GZIP Format) decompression with virtual file pointer support, enabling efficient random access within compressed data. Handles multi-block compression, concurrent decompression, and index-based seeking. This is the core I/O layer for all BAM/CRAM/VCF BGZF files and demonstrates a well-crafted pattern for block-compressed stream reading with positional queries.
A generic interval tree implementation for efficient overlap queries, insertion, and deletion of intervals. Optimized for genomic coordinates with support for overlapping intervals, containment checks, and iteration. The tree is self-balancing (AVL-based) and widely used in the codebase for interval-based filtering and indexing. Could be reused in any project requiring interval management.
Provides external sorting of objects using a temporary file when data exceeds memory. Uses a configurable comparator, codec for serialization, and merges multiple sorted runs. Implements CloseableIterator for consumption. This is a reusable utility for sorting large datasets without loading everything into memory, with a clean API that separates serialization from sorting logic.
Manages a pool of background threads for asynchronous writing to multiple output streams. Uses bounded queues to prevent memory exhaustion, and supports exception propagation from worker threads. The pattern is generic and could be reused for any high-throughput async output scenario, not just sequencing data.
Defines a Bundle abstraction that groups related I/O resources (e.g., data file, index, reference) for a codec. Supports multiple resource types (IOPathResource, InputStreamResource, SeekableStreamResource) and JSON serialization. This is a flexible pattern for decoupling codec I/O from specific storage backends, reusable in any system that needs to manage multiple related file resources.