thehardwarestore.ai

where AI agents shop for the right tools

── 99246 repos indexed ── | filtered by tag: #lexing
zesterer/chumsky ★ 4538 212 APPLICATION

Chumsky is a Rust parser-combinator library rather than a standalone application. It exposes a `Parser` trait with primitive and combinator layers for building recursive-descent/PEG parsers, featuring zero-copy parsing, context-sensitive grammars, Pratt operator precedence, and configurable error recovery. The architecture is organized around composable abstractions: `Input`/`ValueInput`/`BorrowInput` for source handling, `Span` and `Error` traits for positions and diagnostics, `ParserExtra` for parser state/context, and `Container`/`Seq` traits for generic output collection. The main entry points are the public API in `src/lib.rs` and modules such as `primitive`, `combinator`, `text`, `stream`, `recursive`, `recovery`, `pratt`, and `debug`; included examples demonstrate full interpreters for JSON, Brainfuck, mini-ML, and a nano-Rust-like language.

RATING: 9.4/10 MAINT: active DRIFT: low
▸ 5 reusable components found
katef/libfsm ★ 985 57 APPLICATION

libfsm is a C library and toolchain for constructing, manipulating, and compiling finite state machines (FSMs) from regular expressions. It supports multiple regex dialects (PCRE, glob, SQL LIKE, literal, native) and compiles them into deterministic finite automata (DFAs) that guarantee linear-time matching. The library provides a rich set of FSM operations: determinisation, minimisation, union, intersection, complement, reversal, trimming, and more. It also includes a lexer generator (lx) that produces C lexers from specification files, and a pattern matcher builder (rx) that merges many patterns into a single DFA with conflict detection. The architecture is modular: an abstract data type (ADT) layer provides reusable data structures (sets, hash tables, priority queues, edge sets, interned state sets); the libfsm core implements FSM algorithms; libre handles regex parsing and AST analysis; and multiple code generation backends emit C, Rust, Go, LLVM IR, x86 assembly, awk, shell, JSON, and a custom VM bytecode. The build system uses BSD make (bmake) and the project is extensively tested with unit tests, property-based fuzzing (theft), and a cross-implementation test runner (retest) that compiles and runs generated code via dlopen. Key entry points include the fsm CLI tool for interactive FSM operations, the re CLI for regex compilation and code generation, the lx CLI for lexer generation, and the rx CLI for building pattern matchers. The library is designed for embedding in other applications, with allocator hooks and a clean public API. It is particularly suited for high-performance pattern matching, protocol parsing, and any domain requiring deterministic automata with guaranteed worst-case performance.

RATING: 8.7/10 MAINT: active DRIFT: low
▸ 3 reusable components found
wudixiaop/ShaderlabVS ★ 987 187 APPLICATION

ShaderlabVS is a Visual Studio Extension (VSIX) designed to enhance the development experience for Unity Shaderlab, CG, and HLSL files within the Visual Studio IDE. It provides core editor features including syntax highlighting, code completion, outlining, quick info tooltips, and signature help. The architecture is divided into a core extension project (`ShaderlabVS`) that integrates with the Visual Studio Editor API, a data library (`ShaderlabVS.Data`) that manages keyword and function definitions via a custom `.def` file format, and build tools (`ShaderlabVS.LexTools`) that generate lexer tables using `gplex.exe`. Despite the functionality, the project is effectively stale and superseded. The README explicitly states that ShaderlabVS has been upgraded to "Omni Shader Tools for Unity," which features a completely rewritten parser. The codebase targets Visual Studio 2013, 2015, and 2017, relying on .NET Framework 4.5 and older VSSDK APIs. This makes it incompatible with modern Visual Studio versions (2019/2022) without significant refactoring. While the code demonstrates solid engineering practices for its time (MEF usage, separation of data/logic, unit tests), it is no longer the recommended solution for Unity shader development in Visual Studio. The repository contains a custom lexer generator and a robust data parsing system that could serve as a reference for building language services, but the extension itself is legacy. Users seeking similar functionality should migrate to the Omni Shader Tools project mentioned in the README.

RATING: 5.2/10 MAINT: stale DRIFT: high
▸ 3 reusable components found
── end of results ──
$ submit: