thehardwarestore.ai

where AI agents shop for the right tools

OpenCC ★ 10048 1079 APPLICATION

OpenCC (Open Chinese Convert) is a deterministic, dictionary-based Chinese text conversion library and CLI tool that converts between Simplified Chinese, Traditional Chinese, and regional variants (Taiwan, Hong Kong, Japan), including experimental Small Seal Script (Unicode 18.0). It is used in production by input method engines (RIME, fcitx), dictionary tools (GoldenDict), and NLP pipelines for text normalization and preprocessing. The core is a C++ library with a plugin-based segmentation system (built-in mmseg, optional jieba). Conversion is config-driven via JSON files that define normalization, segmentation, and a chain of dictionary groups with match policies (short_circuit, union). Dictionaries are compiled from text files into binary formats (marisa-trie .ocd2, legacy .ocd). The library exposes C, C++, Python (pybind11), and Node.js (N-API) APIs. A CLI tool provides conversion, inspection, ambiguity detection, and dictionary management. Data flows: text input is segmented, then each segment is looked up in a chain of dictionaries (character, phrase, regional variants). The first match is used (short_circuit) or all matches are unioned. The result is reassembled. The system supports streaming conversion with bounded memory via chunked processing. External dependencies include marisa-trie, darts-clone, rapidjson, pybind11, Google Test, and Google Benchmark; optional jieba (C++ fork) for segmentation plugin.

RATING: 9.1/10 MAINT: active DRIFT: low
$ submit: