$ show Tencent/hpc-ops
This repository is a high-performance computing library optimized for large language model (LLM) inference and training on NVIDIA GPUs. It provides accelerated implementations of core operations including attention mechanisms, GEMM, fused Mixture of Experts (MoE), all-reduce with normalization, sampling, and rotary position embeddings. The architecture combines C++/CUDA kernels for performance-critical operations with Python bindings via PyTorch, enabling easy integration into deep learning frameworks. The library emphasizes kernel fusion, efficient memory access patterns, and specialized optimizations for modern GPU architectures (particularly Hopper and Blackwell).
The LLM identified 5 files worth highlighting for reuse:
Implements efficient GPU-to-GPU communication using NVLink multicast operations to reduce communication overhead in distributed settings. Demonstrates advanced knowledge of GPU interconnects with a clean interface for initialization and point-to-multipoint communication. Could be extracted for use in other multi-GPU applications requiring efficient collective operations.
Provides an efficient attention mechanism with KV cache compression through preprocessing, GEMM operations, and Top-P Differential (TPD) mask generation. Features modular design with clear separation of concerns and demonstrates a novel approach to reducing memory bandwidth in attention computation. Could be adapted for other attention optimization techniques in transformer architectures.
Combines RoPE (Rotary Position Embedding) computation with KV cache storage in a single fused kernel, supporting multiple precision formats (BF16, FP8). Demonstrates effective kernel fusion techniques to reduce kernel launch overhead and memory transactions. Self-contained with clear mathematical implementation that could be used in other transformer implementations requiring efficient position encoding.
Implements fused all-reduce operations with RMS normalization, offering both high-throughput and low-latency variants. Demonstrates efficient combination of communication and computation to minimize synchronization overhead. Could be used in other distributed training scenarios where normalization follows collective operations.
Supports efficient batched GEMM with varying problem sizes, including FP8 support and multiple optimization strategies. Demonstrates sophisticated batching techniques for handling heterogeneous workloads in a single kernel launch. Could be used in other batched linear algebra scenarios such as Mixture of Experts or multi-head attention implementations.