$ show krmanik/HSK-3.0
This repository is a curated collection of Chinese vocabulary lists for standardized tests (HSK 3.0, BCT, YCT), enriched with pinyin, zhuyin, part-of-speech tags, frequency data, and English definitions. It includes Python and JavaScript scripts for data cleaning, format conversion (TXT/TSV/XLSX), POS tag correction using SUBTLEX frequency corpus, and pinyin tone conversion. The data is organized by test level and version (2021 and 2025 HSK standards), with multiple output formats suitable for flashcard apps (Anki, Pleco) and language learning tools. The repository is primarily a data asset with supporting ETL scripts, not a production application. It demonstrates solid data engineering practices for language resource curation, though it suffers from some file duplication and hardcoded paths.
The LLM identified 2 files worth highlighting for reuse:
Uses SUBTLEX frequency corpus to correct incorrect part-of-speech tags in the CC-CEDICT database. For each word, it finds the dominant POS from the highest-frequency SUBTLEX row and updates the cedict.db accordingly. This is a reusable pattern for improving dictionary POS annotations using external frequency data, applicable to any language resource with frequency and POS information.
Converts numbered pinyin (e.g., 'ni3 hao3') to accented tone marks (e.g., 'nΗ hΗo'). Handles edge cases like 'ΓΌ', 'vβΓΌ', and multi-syllable words. This is a self-contained, reusable utility for any project needing pinyin normalization, easily extractable and adaptable.