$ show lmdu/pyfastx
pyfastx is a Python C extension library and command-line tool for efficient parsing and random access of FASTA and FASTQ files. It builds SQLite3-based indexes to enable fast sequence retrieval without excessive memory usage, supporting both plain and gzipped files. The core functionality is implemented in C using kseq.h for parsing and zran.c for gzip indexing, with a Python wrapper providing high-level classes (Fasta, Fastq, Sequence, Read) and key collections for filtering and sorting. The included CLI tool offers subcommands for indexing, statistics, splitting, format conversion, subsequence extraction, and sampling. The project includes comprehensive benchmarks comparing against tools like pysam, pyfaidx, and biopython, and is cited in peer-reviewed literature.
The LLM identified 2 files worth highlighting for reuse:
Automates the build process for the C extension by fetching and compiling dependencies (zlib, sqlite3, indexed_gzip) and setting platform-specific compiler flags, enabling reproducible builds across Windows, Linux, and macOS with minimal user configuration.
Implements a SQLite3-based index to store sequence offsets and lengths for FASTA/FASTQ files, allowing efficient random access without loading entire files into memory, and integrates with zran.c for gzip file indexing, demonstrating a scalable pattern for handling large genomic datasets.