$ show opendp/smartnoise-sdk
The SmartNoise SQL/Synth/Eval monorepo is a differentially private (DP) data analysis toolkit. It provides three main packages: `snsql` (DP SQL query rewriting with calibrated noise), `snsynth` (DP synthetic data generation via MWEM, GANs, MST, and other mechanisms), and `sneval` (evaluation of synthetic data quality). The SQL package includes a custom ANTLR-based SQL parser (`SqlSmall.g4`) that rewrites queries to enforce DP, supports multiple backends (Pandas, Spark, PostgreSQL, SQL Server, BigQuery), and tracks privacy budgets via an odometer. The synth package offers a rich set of synthesizers (DPCTGAN, PATECTGAN, MWEM, MST, AIM, etc.) with a consistent `fit`/`sample` API and a `TableTransformer` pipeline for preprocessing. The eval package provides metrics for comparing real vs. synthetic data, including accuracy, bias, and suppression/fabrication counts. The repository includes extensive tests (SQL parsing, mechanism accuracy, synthesizer round-trips) and sample notebooks demonstrating DP workflows.
The LLM identified 6 files worth highlighting for reuse:
Rewrites SQL ASTs to enforce differential privacy: converts AVG to SUM/COUNT, adds clamping, applies noise mechanisms (Laplace/Gaussian), and injects post-processing (clamp, sort, limit). Core of the PrivateReader pipeline.
Differentially private CTGAN implementation using Opacus for DP-SGD. Supports conditional generation via training-by-sampling, with configurable discriminator/generator architecture and DP optimizer settings.
ANTLR4 grammar defining the SQL dialect supported by SmartNoise SQL. Handles SELECT, JOINs, GROUP BY, HAVING, ORDER BY, LIMIT, expressions, and window functions. Generated lexer/parser are checked in; grammar changes require regeneration.
TableTransformer provides fit/transform/inverse_transform for tabular data, with factory methods for column-specific transformers (MinMax, OneHot, Label, Bin, DateTime). Enables consistent preprocessing for synthesizers.
Privacy odometer for tracking epsilon/delta composition across queries. Supports homogeneous and heterogeneous composition, with a `get_privacy_cost()` method for estimating cost without executing a query.
Comparison metrics for evaluating synthetic data quality: accuracy, bias, suppression/fabrication counts, and marginal errors. Used by the Evaluate class to compare real vs. synthetic datasets.