$ account jessevig

[github ↗] 1 repos indexed

bertviz ★ 8161 885 APPLICATION

bertviz is a Python library for interactive visualization of attention in Transformer-based NLP models. It provides three complementary views: Head View (per-head attention patterns), Model View (attention flow between tokens across layers), and Neuron View (query/key interactions within individual attention heads). The library is designed for Jupyter notebooks, rendering D3.js-based interactive HTML directly in cell output. It accepts attention tensors from most HuggingFace models via output_attentions=True, and includes a vendored, modified copy of older HuggingFace transformers code to expose query/key vectors for the Neuron View on BERT, GPT-2, RoBERTa, XLNet, and similar architectures.

RATING: 7.0/10 MAINT: maintained DRIFT: high

#transformer-interpretability #attention-visualization #nlp #jupyter #d3-visualization #huggingface #model-debugging #natural-language-processing #machine-learning #visualization #neural-network #pytorch #bert #transformer #gpt2 #roberta #transformers