Made With ML is an educational data science project that implements an end-to-end ML pipeline for classifying GitHub repositories into categories (NLP, computer vision, MLOps, other) based on their titles and descriptions. The core data lineage flows from raw CSV datasets (dataset.csv, holdout.csv) through text cleaning, tokenization via SciBERT, and stratified splitting into train/val/test sets, followed by distributed training with PyTorch and Ray Train, hyperparameter tuning via HyperOpt, evaluation with slice-based metrics using Snorkel, and serving through a FastAPI/Ray Serve deployment. The project demonstrates MLOps patterns including MLflow experiment tracking, data validation with Great Expectations, behavioral model testing, and CI/CD via GitHub Actions with Anyscale for cloud execution. The codebase is well-structured with clear separation of concerns across data.py, train.py, tune.py, evaluate.py, predict.py, serve.py, models.py, and utils.py. It includes comprehensive tests at the code, data, and model levels. However, the project shows signs of staleness with pinned dependencies from 2023 (Ray 2.7.0, transformers 4.28.1, torch 2.0.0) that have known security vulnerabilities and compatibility issues with current Python versions. The data pipeline components—particularly the CustomPreprocessor, stratified splitting, and text cleaning—are genuinely reusable patterns. The evaluation framework with slice-based metrics and behavioral tests (invariance, directional expectations, minimum functionality) represents solid ML engineering practice. The deployment configuration is tightly coupled to Anyscale's proprietary platform, limiting portability.
DeepOnto is a Python package for ontology engineering with deep learning, centered on ontology alignment (BERTMap) and subsumption inference (BERTSubs, OntoLAMA). It wraps the OWLAPI Java library through JPype to load, query, reason over, and prune OWL ontologies, and uses PyTorch/Hugging Face Transformers to train or apply BERT-based classifiers for mapping prediction and refinement. The package exposes config-driven pipelines (BERTMapPipeline, BERTSubs pipelines, OntoLAMA inference) plus evaluation utilities for OAEI-style benchmarks.
This repository is the official code companion to the book '50 ML projects to understand LLMs' by Mike X Cohen. It contains over 50 Jupyter notebooks organized into chapters (2–7), each exploring a specific concept in large language model interpretability and internals. Topics span tokenization, embeddings, cosine similarity, softmax/logits, loss functions, sampling, hidden state analysis, effective dimensionality, logit lens, attention mechanisms, MLP projections, probing classifiers, and bias measurement. The code is purely educational and experimental—no production services, APIs, or deployable applications. All notebooks are designed to run on Google Colab with minimal setup, using standard libraries (PyTorch, Hugging Face Transformers, scikit-learn, numpy, matplotlib) and pretrained models (GPT-2, BERT, RoBERTa, Pythia). Each project has a 'helper' notebook (incomplete code for students) and a 'solution' notebook (complete reference implementation). The repository is a valuable resource for learning mechanistic interpretability and hands-on LLM analysis, but it lacks modularization, dependency pinning, and test coverage, making it unsuitable for direct reuse in production pipelines.
bertviz is a Python library for interactive visualization of attention in Transformer-based NLP models. It provides three complementary views: Head View (per-head attention patterns), Model View (attention flow between tokens across layers), and Neuron View (query/key interactions within individual attention heads). The library is designed for Jupyter notebooks, rendering D3.js-based interactive HTML directly in cell output. It accepts attention tensors from most HuggingFace models via output_attentions=True, and includes a vendored, modified copy of older HuggingFace transformers code to expose query/key vectors for the Neuron View on BERT, GPT-2, RoBERTa, XLNet, and similar architectures.
This repository is a curated collection of five large-scale Chinese NLP datasets (Wikipedia articles, news, QA pairs, community Q&A, and translation pairs) with descriptions and download links. It functions as a data catalog rather than a software project: there is no executable code, no ETL scripts, and no pipelines. The datasets are hosted externally (Google Drive, Baidu) and require manual download. The repository provides clear formatting examples and suggested use cases, but lacks any automation or reproducibility support such as download scripts, environment definitions, or data processing code. It is a valuable reference for researchers needing Chinese text corpora, but it is not a reusable data engineering artifact.
This repository is a Chinese-language chatbot built with Rasa 1.10.18, a now-deprecated version of the Rasa framework. It implements a simple customer service bot for telecom queries (phone bill and data usage) using a BERT-based NLU pipeline (HFTransformersNLP with bert-base-chinese) and a rule-based dialogue management system with story-driven flows. The codebase is minimal: a single custom action for querying consumption, a few intents and entities, and a set of dialogue stories. The project is functional but lacks tests, CI/CD, or modern Rasa practices. The pinned dependencies (rasa==1.10.18, transformers==2.4.1) are severely outdated, and the code relies on deprecated Rasa components. While the repository has accumulated stars (983) due to its accompanying blog series, the actual code is a basic Rasa template with little reusable craftsmanship. It serves as a learning example for building a Chinese chatbot with Rasa 1.x, but is not suitable for production or reuse without significant modernization.
Bert-VITS2 is a multilingual text-to-speech system based on VITS2 (Variational Inference Text-to-Speech with improved Transformer backbone). It supports Chinese, Japanese, and English synthesis with optional emotion conditioning via pre-trained models (WavLM, wav2vec2-emotion, CLAP). The codebase provides a complete pipeline: data preprocessing (resampling, text normalization, BERT feature extraction), multi-speaker training with distributed support, inference through a Gradio web UI or FastAPI server (hiyoriUI.py), and ONNX export for deployment. The architecture uses separate per-language BERT models for text encoding, a flow-based decoder, stochastic duration predictor, and multi-scale discriminators (including a WavLM-based discriminator). The repository includes numerous versioned subdirectories (oldVersion/, onnx_modules/) for backward compatibility with model versions 1.0.1 through 2.4, resulting in significant code duplication and maintenance overhead. The README explicitly states the project is no longer maintained and recommends the successor project Fish-Speech.