$ show scientist-labs/red-candle
Red-candle is a Ruby gem that provides bindings to the Hugging Face Candle library, enabling Ruby developers to run state-of-the-art language models directly in Ruby without Python or external APIs. The system supports LLMs (Mistral, Llama, Gemma, etc.), embedding models, rerankers, NER models, and vision-language models with hardware acceleration via Metal (Apple GPU) and CUDA (NVIDIA GPU). The architecture features a clean separation between Ruby interface layer and Rust extension. The Ruby layer provides idiomatic APIs (Candle::LLM, Candle::EmbeddingModel, etc.) while the Rust extension handles model loading, tensor operations, and hardware-specific computations using the Candle ML library. Key design patterns include unified model loading via `from_pretrained`, device abstraction for CPU/GPU selection, tokenizer access across all models, structured generation with JSON schema constraints, and a complete tool calling framework with agent capabilities. The project demonstrates strong engineering practices with comprehensive test coverage, extensive examples showing real-world usage (biomedical NER, chat interfaces, reranker benchmarks), and clear documentation. It successfully bridges the Ruby and Rust ecosystems while maintaining performance and usability, making advanced AI accessible to Ruby developers through a consistent, well-abstracted interface.
The LLM identified 3 files worth highlighting for reuse:
Provides a unified interface for device selection (CPU, Metal, CUDA) with automatic fallback logic. Extracts hardware-specific complexity into a reusable pattern that could be used in any Ruby/Rust project needing GPU acceleration. Demonstrates skillful handling of fallbacks and device-specific tensor operations.
Implements structured generation using vocabulary-aware finite state machines to constrain LLM outputs to JSON schemas or regex patterns. This reusable pattern solves the common problem of forcing LLMs to produce valid structured data. Shows non-obvious skill in integrating constraint checking directly into the generation process.
Complete framework for implementing tool/function calling with LLMs, including automatic parsing of model-generated tool calls, execution loops, and result handling. This reusable component encapsulates the complex agent pattern for multi-step reasoning. Demonstrates skill in managing the interaction loop between LLMs and external tools while handling edge cases like tool errors and recursion limits.