$ show ntop/nDPI
nDPI is a production-grade open-source deep packet inspection (DPI) library written in C, developed primarily by ntop.org. It identifies hundreds of network protocols by inspecting packet payloads using static signatures, heuristics, state machines, and machine learning classifiers. The library extracts rich metadata (TLS fingerprints, DNS queries, HTTP headers, etc.), computes flow risks (malware, DGA, obfuscation, anomalies), and supports output serialization in JSON, CSV, or TLV formats. It is designed to be embedded into network monitoring applications such as ntopng, nProbe, and Wireshark (via a Lua extcap plugin), and also ships as a standalone CLI tool (ndpiReader) and Python bindings. The codebase is well-structured with a modular dissector pattern, extensive fuzzing infrastructure, and a comprehensive test suite. While the primary focus is on library reuse, it also contains research-grade components like DGA detection models (scikit-learn/TensorFlow) and time-series anomaly detection, giving it a hybrid character leaning heavily toward a software library.
The LLM identified 5 files worth highlighting for reuse:
A format-agnostic serializer supporting JSON, CSV, and TLV output with dynamic buffer management, JSON string escaping, and snapshot/rollback. The API is key-value pair based and can be extracted and reused in any C project needing lightweight, embedded serialization.
A self-contained statistical analysis library providing sliding window statistics (mean, variance, min/max), entropy, RSI, Holt-Winters, SES/DES forecasting, jitter, outlier detection, and isolation forest. All functions operate on simple data structures (ndpi_analyze_struct, ndpi_bin) and are reusable for any application requiring lightweight time-series or network traffic analysis.
A set of helpers for fuzzing C code, including functions to set allocation callbacks (to simulate allocation failures), a buffer-to-file writer, and a seed generator. This infrastructure is independent of nDPI and can be copied into any C fuzz harness to enable OSS-Fuzz compatible memory-failure testing.
Implements a logistic regression classifier trained on Markov-chain features of packet lengths and inter-arrival times, plus byte-distribution features. The feature extraction functions (ndpi_get_mc_rep_lens, ndpi_get_mc_rep_times) and the parameter-loading mechanism (ndpi_update_params) are modular and could be adapted for other sequence classification problems beyond network traffic.
A wrapper around the Roaring bitmap library providing 32-bit and 64-bit bitmaps with serialization/deserialization, union/intersection/difference, and cardinality operations. The API is simple and can be extracted for any C project needing compact, high-performance bitmap operations.