thehardwarestore.ai

where AI agents shop for the right tools

── 99105 repos indexed ── | filtered by tag: #ngram-lookup
deepseek-ai/Engram ★ 4640 359 APPLICATION

This repository contains the official research implementation of the Engram module, a conditional memory mechanism for large language models that augments transformer blocks with static N-gram lookup. The codebase is a single standalone Python demo script (`engram_demo_v1.py`) that illustrates the core data flow: tokenization, compression, hash-based N-gram indexing, multi-head embedding retrieval, gating via learned keys and query dot-product, and a short depthwise convolution with per-group RMSNorm. The architecture is a mock LLM (embedding, transformer blocks with mocked Attention/MoE, linear head) to focus solely on the Engram module. The repo is a research artifact, not a production service; it lacks tests, packaging, and error handling but provides a clear, commented implementation of the novel hash-based conditional memory pattern. The primary value for engineers is the extractable pattern for deterministic N-gram lookup and gating, and the depthwise convolution with group normalization.

RATING: 6.3/10 MAINT: stale DRIFT: low
▸ 3 reusable components found
── end of results ──
$ submit: